Event Sourcing

1. What is it?
2. Demystify how it could work in practice

Event Sourcing

1. What is it?
2. Demystify how it could work in practice

*still working out the specifics

Databases

We're familar with CRUD based design. Create, Read, Update, and Delete.

• Databases typically store the current state.
• Update/Delete mutates the state.

Databases

We're familar with CRUD based design. Create, Read, Update, and Delete.

• Databases typically store the current state.
• Update/Delete mutates the state.

With the event sourcing pattern pattern...

Instead of mutating state directly, we append every state change as an Event object to an event log.

Databases

We're familar with CRUD based design. Create, Read, Update, and Delete.

• Databases typically store the current state.
• Update/Delete mutates the state.

With the event sourcing pattern pattern...

Instead of mutating state directly, we append every state change as an Event object to an event log.

⭐ The application state at any point in time can be rebuilt by processing the stored events.

Shopping Cart Example

Events

1. Cart created
2. Item 1 added
3. Item 2 added
4. Item 1 removed
5. Shipping information added
6. Make payment

Events

1. Cart created
2. Item 1 added
3. Item 2 added
4. Item 1 removed
5. Shipping information added
6. Make payment

State:

{
  "cart_id": 1,
  "datetime": "2020-11-01T21:34+10:30"
}

Events

1. Cart created
2. Item 1 added
3. Item 2 added
4. Item 1 removed
5. Shipping information added
6. Make payment

State:

{
  "cart_id": 1,
  "datetime": "2020-11-01T21:34+10:30",
  "items": [
    { "item_code": "item 1" }
  ]
}

Events

1. Cart created
2. Item 1 added
3. Item 2 added
4. Item 1 removed
5. Shipping information added
6. Make payment

State:

{
  "cart_id": 1,
  "datetime": "2020-11-01T21:34+10:30",
  "items": [
    { "item_code": "item 1" },
    { "item_code": "item 2" }
  ]
}

Events

1. Cart created
2. Item 1 added
3. Item 2 added
4. Item 1 removed
5. Shipping information added
6. Make payment

State:

{
  "cart_id": 1,
  "datetime": "2020-11-01T21:34+10:30",
  "items": [
    { "item_code": "item 2" }
  ]
}

Events

1. Cart created
2. Item 1 added
3. Item 2 added
4. Item 1 removed
5. Shipping information added
6. Make payment

State:

{
  "cart_id": 1,
  "datetime": "2020-11-01T21:34+10:30",
  "items": [
    { "item_code": "item 2" }
  ],
  "address": "298 Payneham Rd, Payneham SA",
}

Events

1. Cart created
2. Item 1 added
3. Item 2 added
4. Item 1 removed
5. Shipping information added
6. Make payment

State:

{
  "cart_id": 1,
  "datetime": "2020-11-01T21:34+10:30",
  "items": [
    { "item_code": "item 2" }
  ],
  "address": "298 Payneham Rd, Payneham SA",
  "paid": true
}

Events

1. Cart created
2. Item 1 added
3. Item 2 added
4. Item 1 removed
5. Shipping information added
6. Make payment

State:

{
  "cart_id": 1,
  "datetime": "2020-11-01T21:34+10:30",
  "items": [
    { "item_code": "item 2" }
  ],
  "address": "298 Payneham Rd, Payneham SA",
  "paid": true
}

Stores

• "Entity store"
• "Read model"
• "Cache"
• "Materialised View"
• "Projection"

Stores

• "Entity store"
• "Read model"
• "Cache"
• "Materialised View"
• "Projection"

Examples

• An embedded cache. eg. python dict
• Relational database.
• Document database. eg JSONB
• ElasticSearch

Stores

• "Entity store"
• "Read model"
• "Cache"
• "Materialised View"
• "Projection"

Examples

• An embedded cache. eg. python dict
• Relational database.
• Document database. eg JSONB
• ElasticSearch

All of the above. Depends on your application needs.

Stores

• "Entity store"
• "Read model"
• "Cache"
• "Materialised View"
• "Projection"

Examples

• An embedded cache. eg. python dict
• Relational database.
• Document database. eg JSONB
• ElasticSearch

All of the above. Depends on your application needs.

The key difference with an event sourcing design is that entity stores no longer act as the system of record – they could easily get replaced or rebuilt as needed by reprocessing the event log.

Denormalise data to your heart’s content

In databases, it is often considered best practice to normalise data, because if something changes, you then only have to change it one place.

Normalisation makes writes fast and simple, but means you have to do more work (joins) at read time.

https://www.confluent.io/blog/making-sense-of-stream-processing/

Large organisations use stream processing software to implement event sourcing in production:

• Apache Kafka
• Amazon Kinesis

Large organisations use stream processing software to implement event sourcing in production:

• Apache Kafka
• Amazon Kinesis

https://www.confluent.io/blog/event-sourcing-cqrs-stream-processing-apache-kafka-whats-connection/

Why

• Loose coupling
• Read and write performance. CQRS
• Flexibility and agility
• Auditing

End

Mistakes we made adopting event sourcing (and how we recovered) http://natpryce.com/articles/000819.html

https://medium.com/@hugo.oliveira.rocha/what-they-dont-tell-you-about-event-sourcing-6afc23c69e9a

https://www.eventstore.com/blog/event-sourcing-and-cqrs

TODO

• Snapshots