How to leverage manufacturing data with Streamlit and Snowflake

In today's data-driven world, manufacturing firms are constantly seeking ways to harness the power of their data to drive better decision-making and optimize operations. One of the key challenges they face is finding the right tools to visualize and work with their data effectively. In March 2022, Snowflake announced they would be acquiring Streamlit, whose open source framework is used to quickly and easily build web-based data, analytics and AI applications.

The volume and variety of data that manufacturing companies produce make them perfect candidates to take advantage of a new paradigm of data applications presented by this partnership. By leveraging Snowflake’s native application framework and Snowpark with Streamlit applications, organizations can quickly prototype, build and iterate on data applications, shortening the time needed to bring data directly to business users and analysts to make sense of their data. In this blog post, we will explore how Streamlit can serve as an alternative to traditional business intelligence tools for visualizing and working with data, and how it can help manufacturing firms augment their analytics environment.

Custom fit for manufacturers

In an article published in The Manufacturer, Tom St John synthesizes some findings from a virtual round table with manufacturing experts about data and digital transformation. He mentions that off-the-shelf data solutions don’t always provide everything that manufacturers need, and that there are some exciting and valuable opportunities presented by tailored solutions. This is where Snowflake and Streamlit come into play.

Plenty of organizations are already taking advantage of the scale and performance that Snowflake provides with their Data Cloud for integrating data across source systems into one place. Adding Streamlit into this ecosystem unlocks the ability to quickly develop custom web applications for interacting with that data.

One of the benefits of the Streamlit framework is that it doesn’t require all of the frontend-specific skills that traditional web application stacks require, like HTML, CSS or Javascript. Instead, it lays out a suite of Python-based components ready to be implemented into your app. Because Python is a popular language for people who work with processing and analyzing data, Streamlit essentially brings them right into the fold when it comes to building the apps that interact with that same data. This shortens a few of the steps needed between starting to bring data into Snowflake and ending with a custom tool on a user’s screen to make sense of that data.

Another challenge manufacturers face when adopting new data solutions, according to St John, is that, “By the time a new system is rolled out there’s a ‘better or at least a more shiny’ solution on the table.” Developing tools with Streamlit sidesteps this challenge slightly because a big benefit of the framework is rapid prototyping and rapid iteration, making it possible to develop and release new functionalities as your business’ needs grow and evolve.

Different views for different needs

One of the guiding principles that organizations work toward when implementing a Snowflake Data Cloud is providing a “single source of truth” for all areas of their business, ensuring that everyone has the same basis for decision-making and measuring outcomes. A difficult reality for large organizations like many manufacturers is that different stakeholders can have vastly different requirements for what data they need to access and how they need to access it.

For example, a production manager, a quality assurance analyst and a sales director will each require different ways of working with the data coming out of the single source of truth from Snowflake. This is not to say that all they will need is to each see data tables with different columns in them, but rather that each of their roles could require a totally different way of interacting with that data on a page.

The production manager might need a few slider inputs to lay out production-schedule possibilities based on the makeup of different products; the quality analyst might want to be able to tweak the inputs for different statistical analyses of defects; and the sales manager would probably need to aggregate transactions across specific buckets and time frames that they specify. Building out tools for each of these use cases is achievable with components that all come native to the Streamlit framework, making it so users of all different roles are able to use the same platform to work with the data in the method that suits them best.

Connecting data to outcomes

Another insight St John noted from the manufacturing roundtable was that “one of the early learnings in delivering strategies in data and digital programmes was to make sure you have a storyteller in the room, who can convert data-driven ambition into language which resonates and that people understand.” This underscores the importance of being able to speak to both data-specific and business-oriented audiences and provide a common, digestible vision to both sides. Streamlit provides a breadth of elements that can help tie the data that manufacturers generate back to the strategic goals and metrics that those organizations set, spanning a wide array of potential use cases:

• Production planning: Draw from actuals in historical data to adjust production schedules as needed. Streamlit provides the ability to process data from inventory, sales and production line datastores in Snowflake and allow a user to tweak future schedules based on the data in the same interface.
• Quality analysis: Set statistical thresholds for different quality metrics in a Streamlit interface, and visualize the result sets to try pinpointing the root causes of potential production quality issues.
• Self-service reporting: Pull the exact sales data needed out of Snowflake by choosing the fields and measures, setting specific filtering conditions, and picking a time frame. Streamlit can also provide the ability to specify whether the data needs to be granular or aggregated, as well as if the results should be viewed on the screen or downloaded into a spreadsheet.

Falling into the pit of success

Building a data application that addresses as many different use cases as a manufacturer requires could very easily end up in a spaghetti-code mess of one-off Snowflake queries, user interface pages and data visualization components. To avoid this, it’s important to start the project with a focus on organization and modularity so it can grow comfortably as the business requirements and use cases expand over time.

We at Ollion have created a project framework for developing data visualization apps with Streamlit and Snowflake. It includes a few key features that help get projects started off on the right foot:

• Semantic model: A metadata model is defined that represents the Snowflake Data Cloud in terms common to both the business users and the developers. This alleviates the need for developers to write SQL queries by hand and centralizes some of the business logic required across departments.
• Data visualization accelerator: The framework includes a tool for creating the starting points for data visualizations in a “pick and choose” interface built with the data warehouse’s semantic model. It generates the code for you in reusable components and offers suggestions about where to organize the code within your project. This is meant to jump-start the development process for the project and provide modular code assets that can then be further customized as the business logic requires.
• User authentication: Keep a handle on which users within your organization can access the data application.
• Deployment tools: Streamlit does offer a Community Cloud product for deploying applications publicly for free, but our framework also offers help with containerization and deployment on your own infrastructure.

To gain a better understanding of how the combination of Streamlit and Snowflake could help transform your business, contact Ollion to get started with an assessment.