- Streams (collect & process large streams of\ndata in real time), <\/li>
- Analytics(run standard SQL queries against\nstreaming data), <\/li>
- Firehouse(load streaming data into AWS from many\nsources into S3 and Redshift)<\/li>
- Kinesis Streams vs SQS – Kinesis for data\nprocessed at the same time or within 24 hrs by different consumers, data can be\nreused with in 24 hrs<\/li>
- 24 hrs retention<\/li>
- SQS can write multiple queue using fanout, data\ncannot be reused, 14 days retention<\/li><\/ul>\n\n\n\n
High-level architecture of Kinesis Data Streams – <\/p>\n\n\n\n
![\"\"](\"https:\/\/www.testpreptraining.ai\/tutorial\/wp-content\/uploads\/2019\/09\/image-41.png\")
<\/figure>\n\n\n\n- Producers continually push data to Kinesis Data\nStreams<\/li>
- consumers process the data in real time. <\/li>
- Consumers can be custom application running on\nAmazon EC2 or Amazon Kinesis Data Firehose delivery stream<\/li>
- Store their results using AWS DynamoDB,\nRedshift, or S3. <\/li><\/ul>\n\n\n\n
Kinesis Data\nStreams Terminology<\/strong><\/p>\n\n\n\n
- Kinesis Data Stream – A Kinesis data stream is a\nset of shards. Each shard has a sequence of data records. Each data record has\na sequence number that is assigned by Kinesis Data Streams.<\/li>
- Data Record – A data record is the unit of data\nstored in a Kinesis data stream. Data records are composed of a sequence\nnumber, a partition key, and a data blob, which is an immutable sequence of\nbytes. Kinesis Data Streams does not inspect, interpret, or change the data in\nthe blob in any way. A data blob can be up to 1 MB.<\/li>
- Retention Period – The retention period is the\nlength of time that data records are accessible after they are added to the\nstream. A stream\u2019s retention period is set to a default of 24 hours after\ncreation. You can increase the retention period up to 168 hours (7 days) using\nthe IncreaseStreamRetentionPeriod operation, and decrease the retention period\ndown to a minimum of 24 hours using the DecreaseStreamRetentionPeriod\noperation. Additional charges apply for streams with a retention period set to\nmore than 24 hours. <\/li>
- Producer – Producers put records into Amazon\nKinesis Data Streams. For example, a web server sending log data to a stream is\na producer.<\/li>
- Consumer – Consumers get records from Amazon\nKinesis Data Streams and process them. These consumers are known as Amazon\nKinesis Data Streams Application.<\/li>
- Amazon Kinesis Data Streams Application – An\nAmazon Kinesis Data Streams application is a consumer of a stream that commonly\nruns on a fleet of EC2 instances. There are two types of consumers that you can\ndevelop: shared fan-out consumers and enhanced fan-out consumers. <\/li>
- Shard – A shard is a uniquely identified\nsequence of data records in a stream. A stream is composed of one or more\nshards, each of which provides a fixed unit of capacity. Each shard can support\nup to 5 transactions per second for reads, up to a maximum total data read rate\nof 2 MB per second and up to 1,000 records per second for writes, up to a\nmaximum total data write rate of 1 MB per second (including partition keys).\nThe data capacity of stream is a function of the number of shards that you\nspecify for the stream. The total capacity of the stream is the sum of the\ncapacities of its shards.<\/li>
- Partition Key – A partition key is used to group\ndata by shard within a stream. Kinesis Data Streams segregates the data records\nbelonging to a stream into multiple shards. It uses the partition key that is\nassociated with each data record to determine which shard a given data record\nbelongs to. Partition keys are Unicode strings with a maximum length limit of\n256 bytes. An MD5 hash function is used to map partition keys to 128-bit\ninteger values and to map associated data records to shards. When an\napplication puts data into a stream, it must specify a partition key.<\/li>
- Sequence Number – Each data record has a\nsequence number that is unique per partition-key within its shard. Kinesis Data\nStreams assigns the sequence number after you write to the stream with\nclient.putRecords or client.putRecord. Sequence numbers for the same partition\nkey generally increase over time. The longer the time period between write\nrequests, the larger the sequence numbers become.<\/li>
- Kinesis Client Library – The Kinesis Client\nLibrary is compiled into application to enable fault-tolerant consumption of\ndata from the stream. The Kinesis Client Library ensures that for every shard\nthere is a record processor running and processing that shard. The library also\nsimplifies reading data from the stream. The Kinesis Client Library uses an\nAmazon DynamoDB table to store control data. It creates one table per\napplication that is processing data.<\/li><\/ul>\n\n\n\n
AWS Kinesis\nSummary<\/strong><\/p>\n\n\n\n
Data Structures<\/strong><\/p>\n\n\n\n
- The unit of data stored by Kinesis Data Streams\nis a data record. <\/li>
- A stream represents a group of data records. <\/li>
- The data records in a stream are distributed\ninto shards.<\/li>
- A shard has a sequence of data records in a\nstream. <\/li><\/ul>\n\n\n\n
Stream Creation<\/strong><\/p>\n\n\n\n
- When you create a stream, specify the number of\nshards for the stream. <\/li>
- Each shard can support up to 5 transactions per\nsecond for reads, up to a maximum total data read rate of 2 MB per second. <\/li>
- Shards support up to 1,000 records per second\nfor writes, up to a maximum total data write rate of 1 MB per second (including\npartition keys). <\/li>
- The total capacity of a stream is the sum of the\ncapacities of its shards. <\/li>
- Increase or decrease the number of shards in a\nstream as needed. <\/li>
- You are charged on a per-shard basis. Billing is per shard provisioned, can have as many shards as you want. Records are ordered per shard. <\/li><\/ul>\n\n\n\n
Kinesis Data Stream<\/strong><\/p>\n\n\n\n
- Kinesis data stream, is composed of
- a\nsequence number or unique ID of the record within its shard. Type: String<\/li><\/ul>
- partition\nkey -identifies which shard in the stream the data record is assigned to. Type:\nString <\/li><\/ul>
- data\nblob – Data in blob is opaque and\nimmutable so it is not inspected, interpreted, or changed in any way. <\/li><\/ul><\/li>
- When the data blob, the payload before\nbase64-encoding, is added to the partition key size, the total size must not\nexceed the maximum record size of 1 MB. Type: Base64-encoded binary data\nobject.<\/li>
- Use server-side data encryption for sensitive\ndata using Amazon Kinesis Data Firehose but, Kinesis stream as data source is\nrequired, though the data is stored in the Kinesis stream. <\/li>
- When you send data to Kinesis stream, data is\nencrypted using an AWS KMS key before storing it at rest. <\/li>
- When Kinesis Data Firehose delivery stream reads\nthe data from Kinesis stream, the Kinesis Data Streams service first decrypts the\ndata and then sends it to Kinesis Data Firehose. Kinesis Data Firehose buffers\nthe data in memory based on the buffering hints that you specify and then\ndelivers it to destinations without storing the unencrypted data at rest.<\/li>
- In Kinesis, to prevent skipped records, handle\nall exceptions within processRecords appropriately.<\/li>
- For each Amazon Kinesis Data Streams\napplication, the KCL or Kinesis Client Library, uses a unique Amazon DynamoDB\ntable to keep track of the application’s state. KCL uses the Amazon Kinesis\nData Streams application name to create the name of the table so, each\napplication name must be unique.<\/li>
- If Amazon Kinesis Data Streams application\nreceives provisioned-throughput exceptions, then, increase the provisioned\nthroughput for the DynamoDB table. The KCL creates the table with a provisioned\nthroughput of 10 reads per second and 10 writes per second, but this might not\nbe sufficient for application. For example, if Amazon Kinesis Data Streams\napplication does frequent checkpointing or operates on a stream that is\ncomposed of many shards, you might need more throughput.<\/li>
- PutRecord returns the shard ID of where the data\nrecord was placed and the sequence number that was assigned to the data record.\n<\/li>
- Sequence numbers increase over time and are specific\nto a shard within a stream, not across all shards within a stream. <\/li>
- For guaranteeing strict increasing ordering,\nwrite serially to a shard and use the SequenceNumberForOrdering parameter.<\/li>
- For live streaming Kinesis gets ruled out if\nrecord size greater than 1 MB , in that case Kafka can support bigger records.<\/li>
- To use Lambda with Kinesis Streams, create\nLambda functions to automatically read batches of records from Amazon Kinesis\nstream and process them if records are detected on the stream. AWS Lambda then\npolls the stream periodically (once per second) for new records. Only One\nlambda per shard be triggered.<\/li>
- In Kinesis stream, the PutRecordBatch()\noperation can take up to 500 records per call or 4 MB per call, whichever is\nsmaller. Buffer size ranges from 1 MB to 128 MB.<\/li>
- In circumstances where data delivery to the\ndestination is falling behind data ingestion into the delivery stream, Amazon\nKinesis Firehose raises the buffer size automatically so that all data is\ndelivered to the destination.<\/li>
- If data delivery to Redshift fail from Kinesis Firehose , Amazon\nKinesis Firehose retries data delivery every 5 minutes for up to a maximum\nperiod of 60 minutes. After 60 minutes, Amazon Kinesis Firehose skips the\ncurrent batch of S3 objects that are ready for COPY and moves on to the next\nbatch. The information about the skipped objects is delivered to S3 bucket as a\nmanifest file in the errors folder, which you can use for manual backfill. <\/li>
- If data delivery to Amazon S3 bucket fails,\nAmazon Kinesis Firehose retries to deliver data every 5 seconds for up to a\nmaximum period of 24 hours. If the issue continues beyond the 24-hour maximum\nretention period, it discards the data.<\/li>
- Aggregation refers to the storage of multiple\nrecords in a Streams record. Aggregation allows customers to increase the\nnumber of records sent per API call, which effectively increases producer\nthroughput. Aggregation Storing multiple records within a single Kinesis Data\nStreams record while Collection using the API operation PutRecords to send\nmultiple Kinesis Data Streams records to one or more shards in Kinesis data\nstream.You can first aggregate stream record and then send them to stream using\ncollection putrecords() in multiple shard.<\/li>
- Spark Streaming uses the Kinesis Client Library\n(KCL) to consume data from a Kinesis stream. KCL handles complex tasks like\nload balancing, failure recovery, and check-pointing<\/li><\/ul>\n\n\n\n
Amazon Kinesis Data Streams limits<\/strong><\/p>\n\n\n\n
- No upper limit on the number of shards in a\nstream or account may be thousands.<\/li>
- No upper limit on the number of streams you can\nhave in an account.<\/li>
- A single shard can ingest up to 1 MiB of data\nper second (including partition keys) or 1,000 records per second for writes.\nSo, stream with 5,000 shards, it can ingest up to 5 GiB per second or 5 million\nrecords per second. Add the number of shards in the stream using the AWS\nManagement Console or the UpdateShardCount API for more ingest capacity.<\/li>
- Default shard limit is 500 shards for the\nfollowing AWS Regions: US East (N. Virginia), US West (Oregon), and EU\n(Ireland). For all other Regions, the default shard limit is 200 shards.<\/li>
- The maximum size of the data payload of a record\nbefore base64-encoding is up to 1 MiB.<\/li>
- GetRecords can retrieve up to 10 MiB of data per\ncall from a single shard, and up to 10,000 records per call. Each call to\nGetRecords is counted as one read transaction.<\/li>
- Each shard can support up to five read\ntransactions per second. Each read transaction can provide up to 10,000 records\nwith an upper limit of 10 MiB per transaction.<\/li>
- Each shard can support up to a maximum total\ndata read rate of 2 MiB per second via GetRecords. If a call to GetRecords\nreturns 10 MiB, subsequent calls made within the next 5 seconds throw an\nexception.<\/li><\/ul>\n\n\n\n
Creating a\nStream in Amazon Kinesis<\/strong><\/p>\n\n\n\n
A stream can be created by <\/p>\n\n\n\n
- the Kinesis Data Streams console<\/li>
- the Kinesis Data Streams API<\/li>
- the AWS Command Line Interface (AWS CLI).<\/li><\/ul>\n\n\n\n
Steps to create a data stream using the console<\/p>\n\n\n\n
- Sign in to the AWS Management Console and open\nthe Kinesis console at https:\/\/console.aws.amazon.com\/kinesis.<\/li>
- In the navigation bar, expand the Region\nselector and choose a Region.<\/li>
- Choose Create data stream.<\/li>
- On the Create Kinesis stream page, enter a name\nfor stream and the number of shards you need, and then click Create Kinesis\nstream.<\/li>
- On the Kinesis streams page, stream’s Status is\nCreating while the stream is being created. When the stream is ready to use,\nthe Status changes to Active.<\/li>
- Choose the name of stream. The Stream Details\npage displays a summary of stream configuration, along with monitoring information.<\/li><\/ul>\n\n\n\n
![\"\"](\"https:\/\/www.testpreptraining.ai\/tutorial\/wp-content\/uploads\/2019\/09\/image-42.png\")
<\/figure>\n\n\n\nKinesis Data\nStreams Producers<\/strong><\/p>\n\n\n\n
A producer puts data records into Amazon Kinesis data\nstreams. For example, a web server sending log data to a Kinesis data stream is\na producer. <\/p>\n\n\n\n
To put data into the stream following is needed <\/p>\n\n\n\n
- name of the stream<\/li>
- a partition key, it determines which shard in\nthe stream the data record is added to and is as per application logic. Its\ncount is usually greater than the number of shards for evenly distribution<\/li>
- the data blob to be added to the stream. <\/li><\/ul>\n\n\n\n
All the data in the shard is sent to the same worker\nthat is processing the shard. <\/p>\n\n\n\n
Using KPL <\/strong><\/p>\n\n\n\n
- The KPL is an easy-to-use, highly configurable\nlibrary, to write to Kinesis data stream. <\/li>
- KPL writes with an automatic and configurable\nretry mechanism<\/li>
- Acts as an intermediary between producer\napplication code and the Kinesis Data Streams API actions. <\/li>
- Collects records and uses PutRecords to write\nmultiple records to multiple shards per request<\/li>
- Aggregates user records to increase payload size\nand improve throughput<\/li>
- Integrates seamlessly with the Kinesis Client\nLibrary (KCL) to de-aggregate batched records on the consumer<\/li>
- Submits Amazon CloudWatch metrics on behalf to\nprovide visibility into producer performance<\/li><\/ul>\n\n\n\n
Using the Amazon Kinesis Data Streams API \u2013 <\/p>\n\n\n\n
- Develop producers using the Amazon Kinesis Data\nStreams API with the AWS SDK for Java. <\/li>
- Two different operations to add data to a stream\n– PutRecords and PutRecord. <\/li>
- PutRecords sends multiple records to stream per\nHTTP request, <\/li>
- PutRecord operation sends records to stream one\nat a time (a separate HTTP request is required for each record). <\/li><\/ul>\n\n\n\n
Using Kinesis Agent \u2013 <\/p>\n\n\n\n
- Kinesis Agent is a stand-alone Java software\napplication <\/li>
- It easily collects and sends data to Kinesis\nData Streams. <\/li>
- The agent continuously monitors a set of files\nand sends new data to stream. <\/li>
- The agent handles file rotation, checkpointing,\nand retry upon failures. <\/li>
- It delivers all of data in a reliable, timely,\nand simple manner. <\/li>
- It also emits Amazon CloudWatch metrics to help\nyou better monitor and troubleshoot the streaming process. <\/li>
- By default, records are parsed from each file\nbased on the newline (‘\\n’) character. <\/li>
- Operating system must be either Amazon Linux AMI\nwith version 2015.09 or later, or Red Hat Enterprise Linux version 7 or later.<\/li><\/ul>\n\n\n\n
Consumers with Enhanced Fan-Out <\/strong><\/p>\n\n\n\n
- Enhanced fan-out, enables consumers to receive\nrecords from a stream with throughput of up to 2 MiB of data per second per\nshard. <\/li>
- This throughput is dedicated, which means that\nconsumers that use enhanced fan-out don’t have to contend with other consumers\nthat are receiving data from the stream. <\/li>
- Kinesis Data Streams pushes data records from\nthe stream to consumers that use enhanced fan-out. <\/li>
- Therefore, these consumers don’t need to poll\nfor data. <\/li>
- You can register up to five consumers per stream\nto use enhanced fan-out. <\/li>
- If you need to register more than five\nconsumers, you can request a limit increase<\/li><\/ul>\n\n\n\n
Splitting a Shard \u2013 <\/p>\n\n\n\n
- Need to specify how hash key values from the\nparent shard should be redistributed to the child shards. <\/li>
- When you add a data record to a stream, it is\nassigned to a shard based on a hash key value. <\/li>
- The hash key value is the MD5 hash of the\npartition key that you specify for the data record at the time that you add the\ndata record to the stream. <\/li>
- Data records that have the same partition key\nalso have the same hash key value. <\/li><\/ul>\n\n\n\n
Merging Two Shards \u2013 <\/p>\n\n\n\n
- It takes two specified shards and combines them\ninto a single shard. <\/li>
- After the merge, the single child shard receives\ndata for all hash key values covered by the two parent shards. <\/li>
- The shards must be adjacent, for merging. <\/li>
- Considered adjacent, if the union of the hash\nkey ranges for the two shards forms a contiguous set with no gaps. For example,\ntwo shards, hash key range of 276…381 and 382…454, can be merged into a\nsingle shard that would have a hash key range of 276…454.<\/li><\/ul>\n\n\n\n
Kinesis Data\nStreams Consumers<\/strong><\/p>\n\n\n\n
- A consumer, or Amazon Kinesis Data Streams\napplication, reads and processes data records from Kinesis data streams. <\/li>
- To send stream records directly to services like Amazon S3, Redshift, Elasticsearch Service or Splunk, use a Kinesis Data Firehose delivery stream instead of creating a consumer application. <\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"
Amazon Kinesis Data Streams can collect and process large streams of data records in real time. create data-processing applications called Kinesis Data Streams applications. Kinesis Data Streams application reads data from a data stream as data records. use for rapid and continuous data intake and aggregation. Input data includes data from logs, social media, market…<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":2471,"menu_order":26,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"footnotes":""},"categories":[2],"tags":[167,7],"class_list":["post-3183","page","type-page","status-publish","hentry","category-amazon-aws","tag-amazon-kinesis","tag-aws"],"acf":[],"yoast_head":"\n
Amazon Kinesis - Testprep Training Tutorials<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Amazon Kinesis - Testprep Training Tutorials\" \/>\n<meta property=\"og:description\" content=\"Amazon Kinesis Data Streams can collect and process large streams of data records in real time. create data-processing applications called Kinesis Data Streams applications. Kinesis Data Streams application reads data from a data stream as data records. use for rapid and continuous data intake and aggregation. Input data includes data from logs, social media, market...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/\" \/>\n<meta property=\"og:site_name\" content=\"Testprep Training Tutorials\" \/>\n<meta property=\"article:modified_time\" content=\"2020-05-02T05:04:09+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.testpreptraining.ai\/tutorial\/wp-content\/uploads\/2019\/09\/image-41.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"14 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/\",\"url\":\"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/\",\"name\":\"Amazon Kinesis - Testprep Training Tutorials\",\"isPartOf\":{\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#website\"},\"datePublished\":\"2019-09-02T10:55:16+00:00\",\"dateModified\":\"2020-05-02T05:04:09+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.testpreptraining.ai\/tutorial\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AWS Certified Solutions Architect Professional (SAP-C02)\",\"item\":\"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Amazon Kinesis\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#website\",\"url\":\"https:\/\/www.testpreptraining.ai\/tutorial\/\",\"name\":\"Testprep Training Tutorials\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.testpreptraining.ai\/tutorial\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#organization\",\"name\":\"Testprep Training\",\"url\":\"https:\/\/www.testpreptraining.ai\/tutorial\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.testpreptraining.com\/tutorial\/wp-content\/uploads\/2020\/07\/tpt-logo-6.png\",\"contentUrl\":\"https:\/\/www.testpreptraining.com\/tutorial\/wp-content\/uploads\/2020\/07\/tpt-logo-6.png\",\"width\":583,\"height\":153,\"caption\":\"Testprep Training\"},\"image\":{\"@id\":\"https:\/\/www.testpreptraining.ai\/tutorial\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Amazon Kinesis - Testprep Training Tutorials","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/","og_locale":"en_US","og_type":"article","og_title":"Amazon Kinesis - Testprep Training Tutorials","og_description":"Amazon Kinesis Data Streams can collect and process large streams of data records in real time. create data-processing applications called Kinesis Data Streams applications. Kinesis Data Streams application reads data from a data stream as data records. use for rapid and continuous data intake and aggregation. Input data includes data from logs, social media, market...","og_url":"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/","og_site_name":"Testprep Training Tutorials","article_modified_time":"2020-05-02T05:04:09+00:00","og_image":[{"url":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-content\/uploads\/2019\/09\/image-41.png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"14 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/","url":"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/","name":"Amazon Kinesis - Testprep Training Tutorials","isPartOf":{"@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#website"},"datePublished":"2019-09-02T10:55:16+00:00","dateModified":"2020-05-02T05:04:09+00:00","breadcrumb":{"@id":"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/amazon-kinesis\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.testpreptraining.ai\/tutorial\/"},{"@type":"ListItem","position":2,"name":"AWS Certified Solutions Architect Professional (SAP-C02)","item":"https:\/\/www.testpreptraining.ai\/tutorial\/aws-certified-solutions-architect-professional-sap-c01\/"},{"@type":"ListItem","position":3,"name":"Amazon Kinesis"}]},{"@type":"WebSite","@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#website","url":"https:\/\/www.testpreptraining.ai\/tutorial\/","name":"Testprep Training Tutorials","description":"","publisher":{"@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.testpreptraining.ai\/tutorial\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#organization","name":"Testprep Training","url":"https:\/\/www.testpreptraining.ai\/tutorial\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#\/schema\/logo\/image\/","url":"https:\/\/www.testpreptraining.com\/tutorial\/wp-content\/uploads\/2020\/07\/tpt-logo-6.png","contentUrl":"https:\/\/www.testpreptraining.com\/tutorial\/wp-content\/uploads\/2020\/07\/tpt-logo-6.png","width":583,"height":153,"caption":"Testprep Training"},"image":{"@id":"https:\/\/www.testpreptraining.ai\/tutorial\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/pages\/3183","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/comments?post=3183"}],"version-history":[{"count":3,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/pages\/3183\/revisions"}],"predecessor-version":[{"id":3397,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/pages\/3183\/revisions\/3397"}],"up":[{"embeddable":true,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/pages\/2471"}],"wp:attachment":[{"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/media?parent=3183"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/categories?post=3183"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.testpreptraining.ai\/tutorial\/wp-json\/wp\/v2\/tags?post=3183"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- partition\nkey -identifies which shard in the stream the data record is assigned to. Type:\nString <\/li><\/ul>
- a\nsequence number or unique ID of the record within its shard. Type: String<\/li><\/ul>
- Kinesis data stream, is composed of