Leveraging LLMs with LangChain for Supply Chain Analytics — A Control Tower Powered by GPT

By Jessie Hobb On Nov 17, 2023

Leveraging LLMs with LangChain for Supply Chain Analytics — A Control Tower Powered by GPT

Build an automated supply chain control tower with a LangChain SQL agent connected to the database of a transportation management system

(Image by Author)

A Supply Chain Control Tower can be defined as a centralized solution that provides visibility and monitoring capabilities to manage end-to-end supply chain operations efficiently.

This analytical tool enables a Supply Chain department to track, understand and resolve critical issues in real time.

Supply Chain Control Tower with Python [Link] — (Image by Author)

In a prior article, I introduced a solution for an analytics control tower (developed with Python) capable of autonomously generating incident reports.

However, this approach encounters limitations in the range of indicators and reports provided.

How can we enhance this model for a better user experience?

This observation led me to the promising potential of Large Language Models (LLMs) like OpenAI’s GPT in providing tailored analysis based on each user request.

High-Level Concept of the Solution — (Image by Author)

In this article, I will share the first steps of my journey to mastering Langchain with OpenAI’s GPT models and building the ultimate Supply Chain Control Tower.

SUMMARY
I. LLMs with LangChain for Supply Chain Analytics
An exploration of how LangChain and LLMs can revolutionize analytics 
in supply chain management.
1. Scenario: Distribution Process of a Fashion Retailer
The complex distribution operations in a global fashion retail setting.
2. Setting the Stage for Experimentation
Introduce the experimental setup to test a LangChain SQL agent. 
3. Experimental Protocol: Starting Simple
Start with a straightforward query to assess the agent's basic capabilities.
II. Experiments and Key Insights
1. Test 1
Simple Prompt without Template
2. Test 2
Prompt Template with a Context
3. Test 3
Prompt Template with an Improved Context
4. Test 4
Advanced Analysis with an Improved Context
5. Final Test
Create a Chain to Write an Email Report
III. LLMs Shaping the Future of Supply Chain
1. A Simple 'Proof-of-Concept'
2. Continuing the Prototype Development
3. Exploring Broader Applications in Supply Chain

LLMs with LangChain for Supply Chain Analytics

Scenario: Distribution Process of a Fashion Retailer

Imagine you are a data scientist at an international clothing group with a global store network.

Your current project involves assisting the distribution planning manager in monitoring stores’ replenishment process.

Supply Chain Network — (Image by Author)

She is leading a team of planners that manage the inventory of stores worldwide.

The process is simple, when the inventory level reaches the minimum level

Planners create replenishment orders in the ERP with information like item quantities and requested delivery dates
Operational teams at the Warehouse prepare the orders for shipment
Transportation planners organize the delivery to the stores

The pivotal performance indicator is the percentage of orders delivered on time.

Distribution Chain Processes Tracked with Timestamps — (Image by Author)

From the order creation to the store delivery, several timestamps and boolean flags are recorded in the database.

Order Transmission time is recorded in ‘Transmission’
If this is after the cut-off time, ‘Transmission_OnTime’ is set to 0.
Truck Loading time is recorded in ‘Loading’
If this is after the cut-off time, ‘Loading_OnTime’ is set to 0.
Truck arrival at the airport is recorded in ‘Airport_Arrival’
If this is after the cut-off time, ‘Airport_OnTime’ is set to 0.
Aircraft landing at the airport is recorded in ‘Airport_Arrival’
If this is after the cut-off time, ‘Airport_OnTime’ is set to 0.
Truck arrival at the city is recorded by ‘City_Arrival’
If this is outside the store opening window, ‘Store_Open’ is set to 0.

The most important timestamp is ‘Delivery_Time’. It is compared with ‘Expected_Delivery_Time’ to set the value of ‘On_Time_Delivery’.

The initial solution I presented in my previous article is a set of visuals and reports answering operational questions.

Question 1: How many shipments have been delivered with delay?

Question 2: Where are the shipments currently in transit?

The main difficulty in designing this kind of descriptive analytics solution is the balance between complexity and completeness.

If you want to answer all the potential operational questions, your report can quickly become extremely difficult to use.
If you want to keep the report concise, you will not cover the full operational scope.

We are reaching the limits of traditional Business Intelligence (BI) tools that rely on visuals, tables and reports to answer operational questions.

For me, the future of reporting lies in dynamically tailored reports that are uniquely customized to each user’s question and context.

Can we use a GPT model to enhance the user experience by providing tailored outputs to each request?

This is what I am trying to figure out with a simple prototype developed with Python.

Setting the Stage for Experimentation

The setup is simple:

A local database ‘shipments.db’ with a single table ‘shipments’
Langchain version 0.0.325
An OpenAI Key to query GPT models
A Python local environment with LangChain, SQLite and Pandas libraries

High-Level Overview of the Solution — (Image by Author)

The database includes the timestamps and the boolean flags along with shipment IDs, destinations and order amounts.

https://medium.com/media/b586ae7d04bbbd6bd1962ee6afa87fda/href

Thus, the LangChain SQL agent (powered by OpenAI’s GPT model) can access the database, write SQL queries and use the outputs to answer users’ questions.

Experimental Protocol: Starting Simple

Because I wanted to sense the agent’s effectiveness, I started with a simple question.

“How many shipments were delayed in the first seven days of May?”

The right answer is 6,816 shipments.

Target behaviour of the Agent — (Image by Author)

I expect to see the agent creating an SQL query that counts all the shipments from ‘2021–05–01’ to ‘2021–05–07’ with the boolean flag ‘On_Time_Delivery’ = False.

In the next section, we will explore different interaction approaches with the agent and search for the most effective method to provide accurate answers.

Experiments and Key Insights

Now that everything is set up, I can start creating the LangChain agent that will interact with the database.

https://medium.com/media/063e02c1fb00a58033fcc726c800db12/href

I use the AgentType.ZERO_SHOT_REACT_DESCRIPTION,a type of agent designed to operate in a "zero-shot" learning context.

This agent is capable of responding to queries without any prior specific training.

Test 1: Simple Prompt without Template

The initial test involves querying the agent with a straightforward question.

“How many shipments were delayed in the first seven days of May?”

https://medium.com/media/b2138e526fd290da5a99d6dc77c0c4d6/href

[Block 1]: The agent starts by exploring the database with the unique table ‘shipments’.

[Block 1]: Discovery of the database — (Image by Author)

The agent is linking the “shipments delayed” of the question with the table ‘shipments’ of the database.

This initial block will be exactly the same for all the other iterations of our experiment.

[Block 2]: The agent writes the query and provides a wrong answer.

[Block 2]: Query the Database and Provide an Answer — (Image by Author)

Test 1: Target Result (Left) / Test Output (Right) — (Image by Author)

👍 A good point is that the agent used the right date (Order Date) to filter the shipments in the scope.
👎 However, he took the wrong flag to define a delayed shipment.

This can be accepted as we did not explicitly define what is a delayed shipment.

Test 2: Prompt Template with a Context

To improve the answer, I can use a prompt template with a context.

https://medium.com/media/a326c8f8ac31e6a63e86754cb34a8d50/href

I wanted to keep the context minimal as we may not always know what users will ask.

context_explanation = """
As a supply chain control tower manager, my role involves overseeing the logistics network and ensuring that shipments are processed efficiently and delivered on time. 
The 'shipments' table in our database is crucial for monitoring these processes. It contains several columns that track the progress and timeliness of each shipment throughout various stages:
- 'Order_Date' and 'Expected_Loading_Date' provide timestamps for when an order was placed and when it was expected to be loaded for departure.
- 'Expected_Delivery_Time' is a timpestamp defining when the shipment is expected to be delivered
- 'Loading_OnTime', 'Airport_OnTime', 'Landing_OnTime', 'Transmission_OnTime' are boolean values indicating whether the shipment was processed on time at various stages. If any of these are False, it implies a delay occurred, potentially causing the shipment to miss its cut-off time and be postponed to the next day.
- 'Store_Open' indicates if the truck arrived at the store before closing time. A False value here means the delivery must wait until the next day.
- 'On_Time_Delivery' is a critical indicator of our service level, showing whether a shipment arrived by the requested delivery time.
Understanding the data in these columns helps us identify bottlenecks in the shipment process and take corrective actions to improve our delivery performance.
"""
input_question = "How many shipments were delayed in the first seven days of May?"

💡 Observation: The context is a high-level presentation of the role of a control tower manager and the content of the database.

[Block 2]: The agent writes the query and provides a wrong answer.

Test 2: Target Result (Left) / Test Output (Right) — (Image by Author)

👎 The agent has a better understanding of the intermediate flags but still misses the point.

This definition of a delayed shipment is not illogical but does not match the operational reality.

💡 Observation: A shipment can be on time even if it has one or several flags at zero. Only the ‘On_Time_Delivery’ flag can determine if a shipment is delayed.

Definitions of cut-off times linked to the flags — (Image by Author)

🙋‍♀️ To be fair to the agent, it’s not a definition that someone could easily guess.

Therefore, I should probably include an explicit definition of a ‘delayed shipment’ in the context.

Test 3: Prompt Template with an Improved Context

I improved the context with an additional sentence.

‘A shipment is considered delayed if ‘On_Time_Delivery’ is false.’

And as expected, the result is good

👋 The agent took the right flag to define delayed shipments.

What if we ask for an advanced analysis?

A delay can be due to various reasons that are captured by the different flags included in the dataset.

Delivered Shipments (Top: On Time, Bottom: Late) — (Image by Author)

Our control tower team would like to get a reason code for each shipment that was not delivered on time.

In this company, the reason code is defined by the list of all intermediate flags that are False.

For instance, if a shipment is delayed :

‘On_Time_Delivery’ is False
‘Transmission_OnTime’ and ‘Loading_OnTime’ are False.
The reason code is then ‘Transmission_OnTime, Loading_OnTime’.

Test 4: Advanced Analysis with an Improved Context

Let’s include an additional statement

‘The reason code of a delayed shipment is defined by the list of all flags that are 0 for this shipment.’

Thus, I can challenge the agent with a new question:

Provide the total number of shipments delayed in the first seven days of May and its split by reason code.

Unfortunately, the agent was not able to compute the right definition of a reason code.

After multiple iterations, I discovered that the agent needed some guidance.

Therefore, I revised the question

Please create the column ‘Reason_Code’ based on the defintion. Then, provide the total number of shipments with delayed delivery in the first seven days of May and the split by reason code.

The output now matches the definition of a reason with a complete analysis of the late delivery root cause.

Can we use this output to send a report by email?

As a final exercise, I wanted to create a chain to ask the agent to write an email using this output.

Final Test: Create a Chain to Write an Email Report

Creating chains with LangChain involves combining multiple agents to perform a sequence of tasks with each task using the output of the previous one.

Agent 1: SQL Query Agent
This agent interprets the user’s question, formulates an SQL query and retrieves the data from the database.
Agent 2: Email Composition Agent
This agent takes the processed data from the SQL Query Agent and composes a coherent and informative email.

We asked Agent 2 to write an email for me (Samir Saci, Control Tower Manager) to the operational director Jay Pity.

[Block 3]: Use the output of Agent 1 to write an email — (Image by Author)

💡 Observation: For an unknown reason, the agent decided to split the number of delayed shipments by day.

[Block 4] Email Output— (Image by Author)

The output is an email summarizing the query results

The agent includes an additional analysis before concluding the email.
The tone is formal and fits with the context of logistic operations management.

The output can be used to automatically send an email using the SMTP library of Python.

💡 What did I learn?

This simple experiment with LangChain SQL Agents taught me that…

Agents are not omniscient. Therefore, specific business definitions must be explained in the context.
Even with a good context, the agent may need guidance to provide the right output.
Several agents can be linked in chains to perform advanced tasks.
Because the agent sometimes needs guidance, we probably have to train users to prompt engineering.

The main challenge is to provide the right context to ensure that the agent can answer all the questions generated by users.

📝Addition Comment: I was quite surprised to see myself using Chat GPT-4, which was quite supportive, to help me improve the context for the prompt template I was using with its ‘little brother’ GPT-3.5 Turbo.

LLMs Shaping the Future of Supply Chain

A Simple ‘Proof-of-Concept’

As I am just starting this exciting journey, I actively seek your comments and observations on the approach I’ve shared in this article.

The initial results promise a transformative future with ‘self-service’ databases enhanced by the capabilities of Large Language Models (LLMs).

LangChain Agents Linked to Multiple Data Products — (Image by Author)

This solution, particularly beneficial for companies implementing data meshes, can be used to connect users directly to data products through a responsive interface enhanced by the power of generative AI.

Users are not using our dashboards. Why?

It enables users to conduct complex analyses through natural language, disrupting our current dashboard-based data interaction.

Continuing the Prototype Development

The conclusion of these initial tests is quite positive.

However, I still have some tests to perform before officially finalizing this proof-of-concept

Enrich the dataset with shipments in transit and cancelled orders
Test how the model is dealing with missing data
Connect the agent with several databases and test how it can manage multiple data sources to answer a question

I would not deploy it in production without user acceptance tests to discover what kind of questions users would ask (and monitor the agent's behaviour).

I will share my future experiments in future articles, feel free to follow me on Medium if you’re interested.

Exploring Broader Applications in Supply Chain

As a Supply Chain Data Scientist, my experimentation doesn’t stop here.

I’m eager to explore other applications of LLMs within the realm of Supply Chain Analytics.

These include integrating LLMs with optimization models:

👬📈 Supply Chain Digital Twins
Application: The agent would help users trigger simulations with scenarios formulated using natural languages.
(Users can ask: “What if we move the central warehouse in Italy?”)
🔗🍃 Sustainable Supply Chain Network Design
Application: Users could create optimization models by formulating the objective and the constraints using natural languages.
(Users can ask: “I would like to create a network of factories to deliver the French market that meets the demand while minimizing CO2 emissions.”)
🏭🍃 Sustainable Sourcing: Select the Best Suppliers
Application: Purchasing teams could formulate their green initiatives using natural languages and see the impact on the costs.
(Users can ask: “We would like to estimate the costs of only selecting carbon-neutral factories to source this SKU.”)

We can also use our agent to improve data quality or support the audit of the data used for strategic reports:

📉📄 ESG Reporting: Environmental, Social and Governance Reporting
Application: Automate the audit of the data used to build the report.
(Auditors can ask: “Could you retrieve the utility bills used to calculate the energy consumption of the plant XYZ?”)
📉✅ What is Data Quality?
Application: Use our agent to challenge or support methodologies that ensure Accuracy, Consistency and Completeness of the Data.
(Users can ask: Could you analyze the number of shipments delivered last year with a missing status?)

Each of these areas holds immense potential for leveraging generative AI to deploy ‘analytics-as-a-service’ solutions in your company.

If you share this enthusiasm, feel free to give some suggestions in the comments section!

About Me

Let’s connect on Linkedin and Twitter, I am a Supply Chain Engineer using data analytics to improve logistics operations and reduce costs.

If you are interested in Data Analytics and Supply Chain, have a look at my website

Samir Saci | Data Science & Productivity

💡 Follow me on Medium for more articles related to 🏭 Supply Chain Analytics, 🌳 Sustainability and 🕜 Productivity.

Leveraging LLMs with LangChain for Supply Chain Analytics — A Control Tower Powered by GPT was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.