Kinesis is that fine for Kafka Loader

#1

Kafka or Kinesis for Kafka Loader? Is there a preference or any issues

#2

By “Kinesis for Kafka” do you mean Amazon Kinesis Source Connector for Confluent Platform? If not, could you please clarify?
Currently TigerGraph can fetch streaming data from Kafka topics only. How the topics are fed with information is not relevant from TigerGraph’s point of view.
On the other hand, Kinesis Data Streams has Python, Java, Node.js, Ruby and .NET client libraries, so should be possible to develop a consumer that feeds data into TigerGraph (via the REST++ API). E.g. we already have a Python connector/wrapper called pyTigerGraph that could be used for this. I am not sure about the performance, though. Clearly, reading directly from a Kafka topic (in parallel fashion) is the most efficient way.

#3

HI, What I mean is Tigergraph has a Kafka data loader. https://docs.tigergraph.com/dev/data-loader-user-guides/kafka-loader-user-guide
I wanted to know rather than Kafka (confluence) can we use Amazon Kinesis. If so is there any known issues?
Appreciated

#4

Hi, As I wrote above, Kinesis is not currently supported; only Kafka. Kinesis can be integrated through Amazon Kinesis Source Connector for Confluent Platform or via a custom app using one of the client libraries (Python, Java, Node.js, Ruby or .NET) and the REST++ API.

#5

Thanks. Yep understand it can built I just needed to confirm the connector was not support by kinesis.

#6

If you need information or assistance with Kinesis integration, please feel free to contact me, I am very interested in all kinds of (analytical ecosystem) integration (involving TigerGraph), I would be happy to help.