AWS/Kinesis
- Streaming fully managed service provided and run by AWS, only pay for use.
- It retains data for 24h by default can increase up to couple of days
Shards
- Records collected in shard (group of records)
- Shard states [ OPEN, CLOSED, EXPIRED]
- One shard capacity 1MB/s in, 2MB/s out, 1000Put/s
- Records in shard limited to 1MB
Retrieve data from Kinesis shards
- API
Kinesis Client Library - The KCL is a Java library