AI for Predicting the Number of Visitors

A Scandinavian supermarket chain with 80+ supermarkets all over the region. 

The daily count of supermarket visitors depends on many factors: day of the week, time of year, upcoming or past holidays, size of the settlement, etc.  

It is very important for the administration of each store to plan in advance the number of employees working on the corresponding days. Too many employees at work lead to profit loss due to excess wages; not enough employees mean lost sales and dissatisfaction of visitors.  

Besides, until recently, the schedule was drawn up manually by store managers, and it took a lot of time to coordinate work assignments, as the manager had to call the employees by phone. The company's management decided to automate the planning of employee work schedules and use AI to forecast demand.  

The challenge of the task is that forecasting systems can usually make high-quality forecasts for 1-4 weeks, while the country's legislation requires employers to inform employees of their schedule at least 4 weeks in advance. This means that the client needs to forecast demand 8 weeks ahead but with the same quality as 1-4-week forecasts, i.e. with 90% accuracy. 

Our tasks also included application deployment automation using a Kubernetes cluster. 

Predicting the number of visitors using AI

At first, we tried regular time series models, such as ARIMA and Prophet. They demonstrated good generalisation ability*, but still did not provide the required accuracy when forecasting for 8 weeks.

Looking for a better solution, we switched to models using the boosting method, i.e. creating an ensemble of weak (poorly predictive) models to build a strong final model. The advantages of the method include resistance to noise and incorrect data and efficiency when working with small amounts of data. Boosting models are widely used in various fields where high forecasting accuracy is important.

The best choice in our case was the open-source CatBoost model, since it allows working effectively with categorical data and using many hyperparameters** to fine-tune the model. In addition, CatBoost ensures high training speed and minimises the risk of overfitting.

We selected the input data necessary to obtain the result of the required accuracy:

• the sales data from 12 months ago,
• the trend for 12 weeks,
• the trend for 8 weeks,
• a feature for taking into account movable holidays.

Since the model only works for the first year, there are cases where the client knows from personal experience that the model's forecast needs to be adjusted for more complex logics than just last year's data. For example, for holidays with a fixed date, demand may depend on the day of the week on which the holiday falls. For movable holidays (Easter and others), the increase in demand may greatly depend on the season in which they occur. In these cases, the model is adjusted or retrained so that the new forecast takes into account additional information.

The UI design was made by Software Country's designer

In addition, we clustered stores by behaviour factor, i.e. grouped stores where the behavior of visitors follows similar patterns. For example, in large cities the influx of visitors on peak dates is much stronger than in small towns, where the number of visitors is always limited by the population of the town. Using data from a specific cluster of stores in the forecasting model further improves the accuracy of predictions.

Web application for work schedule drawing automation

The resulting forecast is used to create a work schedule for employees.

To automate the work assignments approval process, in addition to the standard desktop application, we designed and developed from scratch a small mobile PWA application*** for employees. Through this application, each employee can specify available working hours, and the manager will then see this information when planning. After creating a schedule, the employees receive notifications and see their shift schedules in the app. If there is a need to change the approved plan (for example, if an employee is sick), the app sends notifications to all available employees, and they can take on additional work. Thus, the manager no longer needs to call employees one by one.

The staff planning application

The web application informs, based on a forecast, how many employees should come to work on a certain day/time. In the future, it is planned to automate drawing up individual employee schedules. In addition to planning, the application allows viewing various reports: the number of visitors, the efficiency of each employee, etc. The application is also integrated with the visitor counting system and other internal information systems.

Vue 3 was chosen for application development because it is a popular, actively developing framework for creating user interfaces. It allows you to build a user-friendly and productive interface with minimal costs. C# is used for backend development because it allows you to maintain compatibility with both Linux and Windows.

Application deployment automation using a Kubernetes cluster

Previously, there was a separate pipeline for each of the four client's applications, which had to be launched independently of the others. Since the application updates on the production environment were made weekly, and on the test environment daily or more often, this took quite a lot of time. Instead, we created a universal configurable pipeline with a single interface with checkboxes for selecting which applications need to be updated. This allows significant savings of the DevOps specialist's worktime, as deployment of all applications takes ~15 minutes.

The project uses GitOps method and Flux CD tool.

Currently, the model provides forecasts of the required accuracy (90%) for weeks without holidays. During holiday weeks, the model is adjusted if necessary; this is especially true for movable holidays since the demand for them depends on the date (season) on which the holiday falls. At the end of the annual cycle, it will be possible to evaluate the overall results of the model's work.

Deployment automation has been successfully implemented, the time for releasing updates has been reduced several-fold.