Using the FLaNK Stack for edge ai (apache mxnet, apache flink, apache nifi, apache kafka, apache kudu) lightning talk. Quick talk on running edge ai data pipelines with minifi, nifi, kafka, flink and kudu in any platform at scale.

1. Using the FLaNK Stack for Edge AI (Apache
MXNet, Apache Flink, Apache NiFi, Apache
Kafka, Apache Kudu) - Lightning
Timothy Spann
Principal DataFlow Field Engineer

2. 2
Speakers
Tim Spann
Principal DataFlow Field Engineer
@PaasDev
DZone Zone Leader and Big Data MVB;
Princeton NJ Future of Data Meetup;
https://github.com/tspannhw
https://www.datainmotion.dev/
https://www.flankstack.dev/

3. 3
Welcome to Future of Data - Princeton
@PaasDev
https://www.meetup.com/futureofdata-princeton/
From Big Data to AI to Streaming to Containers to
Cloud to Analytics to Cloud Storage to Fast Data to
Machine Learning to Microservices to ...

4. https://github.com/tspannhw/MmFLaNK
https://www.datainmotion.dev/2019/11/introducing-mm-flank-apache-flink-stack.html

5. MiNiFi Agents

6. Apache MXNet Native Processor for Apache NiFi
Using Java API for Apache MXNet
https://github.com/tspannhw/nifi-mxnetinference-processor
https://community.cloudera.com/t5/Community-Articles/Apache-NiFi-Processor-for-Apache-MXNet-SSD-Single-Shot/ta-p/249240
https://www.datainmotion.dev/2019/12/easy-deep-learning-in-apache-nifi-with.html

7. DJL Wrapped Apache MXNet Processor for Apache NiFi
Using Java API for DJL Wrapping Apache MXNet
https://www.datainmotion.dev/2019/12/easy-deep-learning-in-apache-nifi-with.html
https://github.com/tspannhw/nifi-djl-processor

8. https://www.datainmotion.dev/2020/05/flank-low-code-streaming-populating.html

9. 9
Apache Flink SQL DEMO
HADOOP_USER_NAME=hdfs hdfs dfs -mkdir /user/admin
HADOOP_USER_NAME=hdfs hdfs dfs -mkdir /user/root
HADOOP_USER_NAME=hdfs hdfs dfs -chown root:root /user/root
HADOOP_USER_NAME=hdfs hdfs dfs -chown admin:admin /user/admin
HADOOP_USER_NAME=hdfs hdfs dfs -chmod -R 777 /user
flink-yarn-session -tm 2048 -s 2 -d
flink-sql-client embedded

10. Resources
● https://www.datainmotion.dev/2020/05/flink-sql-preview.html
● https://www.datainmotion.dev/2020/05/time-series-analysis-dataflow.html
● https://github.com/tspannhw/meetup-sensors/blob/main/flink-sql/democdf.sh
● https://www.slideshare.net/bunkertor/time-series-analysis-dataflow
● https://www.datainmotion.dev/2020/07/flank-in-cloud-huge-cloudera-data.html

11. 11
TH N Y U