The retail challenge: Shoplifting on the rise
In Australia, shoplifting has become a significant concern for retailers, with losses totalling $9 billion annually, per the Australian Retailers Association (ARA). This alarming figure highlights the urgent need for innovative solutions to protect businesses, while maintaining a seamless shopping experience for customers and maintaining their privacy. Traditional surveillance systems often fail to detect suspicious behaviours in real time, leaving retailers vulnerable to theft.
What if security systems could detect and understand actions, such as walking, standing, running or concealing items, in real time?
Our breakthrough: AI-powered action recognition
Leveraging cutting-edge, deep learning technologies, like multi-object tracking, Video Transformers, and 3D CNN, our system goes beyond traditional surveillance. It doesn’t just see—it understands. The event-based alerting solution leverages state-of-the-art, in-house-built AI models, meticulously trained over ~2800 hours of heterogeneous retail footage. This diverse dataset includes scenarios from various retail environments, covering different layouts, lighting conditions, customer behaviours, and shoplifting tactics.
The extensive training ensures the system detects everyday actions and accurately distinguishes between normal shopping activities and suspicious behaviours. This versatility makes our solution adaptable to the unique challenges retailers face in any setting.
Our solution is built on a foundation of two critical AI capabilities:
1. Object detection:
- Our system tracks individuals across frames with unparalleled accuracy. Every detected person is assigned a unique ID, ensuring consistent monitoring throughout their time in the store.
2. Video action recognition:
- Powered by advanced video transformers and 3D convolutional neural networks (3D-CNNs), the system analyses a person’s movement over time to classify actions. It doesn’t just look at one frame; it learns from sequences of frames to identify activities like browsing, picking up items, concealing objects, or exiting without paying.
Why our solution stands out
- Our system operates at an impressive >20 frames per second (FPS), ensuring seamless real-time action detection. No delays, no missed actions.
- Over 3,000 hours of training data across diverse retail scenarios, ensuring high accuracy in detecting suspicious behaviours, while understanding normal shopping activities.
- With our MLOps pipeline, the system continuously learns and improves. The entire process is automated, from model updates to deployment, ensuring that the system stays ahead of evolving shoplifting tactics.
- Our system recognises up to 14 unique actions that are highly relevant in retail settings:
- Normal behaviours: Walking, browsing, picking up items, checking price tags, paying.
- Suspicious behaviours: Concealing items, loitering, pocketing items, switching price tags, avoiding staff.
- Critical actions: Running, quick grabbing.
The innovation behind the scenes
- Low latency: Action recognition happens within milliseconds, ensuring real-time detection without disrupting store operations.
- Edge AI: Our solution processes data directly on the premises, reducing reliance on internet connectivity and ensuring privacy.
- Customisable insights: Retailers can tailor action classes to suit their unique needs, whether detecting specific behaviours or understanding customer flows.
Seamless integration and intelligent query capabilities
Our action recognition system is designed to detect shoplifting behaviours and integrate seamlessly into existing alerting systems and workflows. With built-in APIs for real-time alerts, the system can notify store personnel or security teams when a suspicious action is detected. Furthermore, it supports integration with law enforcement agencies, enabling evidence sharing and incident reporting to help prosecute repeat offenders effectively.
But we didn’t stop there. By leveraging large language models (LLMs), our solution offers a human-readable query interface, making it easier for non-technical users to interact with the system. For instance, users can ask questions like:
- “When was the person in a green shirt last seen near the electronics section?”
- “Show me all actions flagged as ‘concealing’ in the past 24 hours.”
- “How many customers walked out without paying today?”
- “Highlight individuals who loitered for more than 10 minutes near the high-value items section.”
This intelligent querying capability enhances the system’s usability and empowers store managers and security teams to make informed decisions quickly.
Privacy-first action recognition with anonymisation
While our action recognition system delivers unmatched accuracy and real-time insights, it is built with privacy. We adhere strictly to local data protection laws and implement industry-standard practices to ensure that no personally identifiable information (PII) is stored or misused.
To maintain compliance with privacy regulations, such as the Australian Privacy Principles (APPs) and GDPR where applicable, the system:
- Anonymises all video data: Individuals are tracked based on unique IDs, not personal characteristics.
- Processes data locally: Where required, sensitive data is processed and stored on-premises to minimise exposure.
- Follows ethical AI standards: All actions are logged transparently, and the system ensures accountability in every decision.
By integrating privacy-by-design principles, our system empowers retailers to enhance security, while respecting customer rights, fostering trust between businesses and their clientele.
The impact: Smarter security for Australian retailers
By combining AI-driven action recognition with object detection, our system provides retailers with a game-changing layer of security. It’s not just about catching shoplifters—it’s about creating a smarter, safer, and more efficient retail environment.
Imagine this: A customer attempts to conceal an item under their clothing. Within moments, the system flags the behaviour, highlights the individual on the live feed, and notifies store personnel. All this happens before they leave the store, minimising losses and ensuring proactive intervention.
Why this matters
With shoplifting contributing billions of dollars in retail shrinkage yearly, this technology is more than a tool—it’s necessary. Our action recognition system helps retailers:
- Reduce theft.
- Improve operational efficiency.
- Protect employees and customers.
Looking ahead: The future of retail AI
As the retail landscape evolves, so do the challenges. By integrating real-time AI capabilities, retailers can stay ahead of the curve. Our system isn’t just about solving today’s problems—it’s about preparing for tomorrow’s opportunities.
Let’s make retail safer, smarter, and more secure.
Contact us for more details or to explore how this technology can transform your retail operations.
