Solutions using Data Visualization

Check out solutions that use Data Visualization here. Interested in something else? See the full list of technologies here.

Build a Data-Intensive, Full Stack App from Start to Finish

Project screenshot

The strength of a full-stack data engineer is the ability to bring the entire stack together, coordinating all of your microservices and features into a single product. Any developer can slap a new feature onto your project, but if new features aren't seamlessly integrated into your project as a whole, they will run inefficiently and slow down future development. See how everything can work together, from data pipeline to web app to data visualization and user-facing search functionality. View on Github

ETL from Cassandra using Spark

Project screenshot

Cassandra DB performs writes fast and leaves read-heavy work to 3rd-party integrations. For example, Elassandra solves this with Elasticsearch and Datastax solves this with Solr and Spark (or even Graph depending on the use case). Of course, we could also integrate Cassandra with these same tools using open source connectors and drivers. Check out an example of how to extract your Cassandra into Spark for an ETL pipeline. View on Github

Manage your Cassandra Logs using the ELK Stack

Project screenshot

Distributed apps quickly get to the place where trying to debug using tail -f becomes untenable. However, ignoring your logs isn't an option: to quote Jay Kreps' book I Heart Logs, "the humble log is an abstraction that lies at the heart of many systems, from NoSQL databases to cryptocurrencies. Even though most engineers don’t think much about them,...logs are worthy of your attention." The ELK Stack (Elasticsearch, Logstash, and Kibana) is a go-to tool for managing your logs and making them help you rather than just taking up hard drive space. Unfortunately, it does not yet have out-of-the-box log processing or dashboards for Cassandra. Check out a way to extract meaningful information from your Cassandra logs here. View on Github

Connect Django Web Server to Google Cloud APIs

Project screenshot

Google provides several powerful APIs which can greatly enhance your app if you are able to take advantage of them. See one such example here, where we upload audio files into Google Speech Recognition API to generate transcripts, using a Python Django backend. View on Github