By Anton Lukin, Sr. Solutions Architect – AWS
By Artem Kobrin, Head of Cloud & Partner – Neurons Lab
| NeuronsLab |
 |
Audits are critical to robust cybersecurity defenses, requiring businesses to regularly assess their information systems for vulnerabilities and compliance with industry standards. The process is essential to identifying threats and ensuring regulatory compliance, but as it involves meticulously reviewing vast amounts of documentation and system configurations, it places a significant burden on cybersecurity teams.
The complexity and volume of the data involved can lead to extended audit durations, delayed threat detection, and increased operational costs. According to Drata’s 2023 Compliance Trends Report, organizations spend, on average, 4,300 hours annually to achieve or maintain compliance – a tremendous amount of time that could potentially be saved with the application of generative AI.
In this post, we outline how Neurons Lab, an AWS Advanced Tier Services Partner and AWS Marketplace Seller with Competencies in Generative AI, Machine Learning, and Healthcare, collaborated with Peak Defence to automate manual compliance processes accurately.
Peak Defence provides information security compliance consulting services, helping clients meet standards and frameworks like ISO 27001, NIST, and SOC2. Neurons Lab is an AI consultancy that provides end-to-end services for mid- and large-sized companies.
Peak Defence’s security audit process and RFP response management were transformed through automation using Amazon Bedrock, Amazon Sagemaker, and Anthropic Claude 3.
Challenges: Increased Demand and Competition
Faced with increased demand, achieving compliance, managing information security systems, and responding to RFPs significantly drained Peak Defence’s employee time and resources, limiting the company’s scalability.
In addition to unlocking new capacity, Peak Defence sought to evolve its offerings in order to remain a market leader in the competitive cybersecurity space. Neurons Lab worked with Peak Defence to implement an AI platform on Amazon Bedrock that leverages LLMs (large language models) to automate manual compliance processes.
Peak Defence’s end goal with the project was to fuel its transition from consulting into a SAAS platform for customers. AI has enabled them to transfer and scale their knowledge out for the customers, creating better security and permitting customers to get clarity and act quickly in a sea of data and security threats.
The reason behind using AWS services, in particular, was to retain control over the customer data and ensure it doesn’t end up in an “uncontrolled” AI learning environment, a major concern in the cyber security world.
Solution: Pioneering Cybersecurity with Generative AI
Several architectural considerations are critical to achieving a robust, scalable, and secure platform. These considerations span selecting AI models based on performance and adaptability, designing the architecture to support scalability and efficiency, and integrating advanced security measures to protect sensitive data.
This solution’s document ingestion process is designed to integrate company policy documentation into a centralized knowledge base seamlessly.
Leveraging various SaaS platforms and enterprise repositories, such as Jira, GitHub, and SharePoint, captures a broad spectrum of organizational knowledge. This data is then processed through LLM Embedding Models, including Amazon Titan Embeddings G1.
As a new way to search for and manage documents, Peak Defence’s AI automation platform now uses Qdrant vector and keyword search.
Vector search is especially useful for understanding content in different languages, finding documents with similar meanings, and searching through short texts with typos or where the meaning isn’t clear without context. However, keyword search is still used to find exact phrases, like security standard names or specific terms, ensuring nothing gets missed.
Overview
Peak Defence’s AI automation platform leverages Claude 3 and Amazon Titan, which is crucial for automating compliance audits and crafting responses to RFPs. These AI models produce human-like text and offer multilingual support, making the solution highly adaptable.
A serverless architecture powered by AWS Step Functions and AWS Lambda ensures operational efficiency and scalability. This setup allows for dynamically managing fluctuating workloads, which is crucial for handling audits and RFPs in a timely manner. AWS Step Functions streamline the orchestration of microservices and data pipelines, while AWS Lambda, integrated with the LangChain framework, offers scalability.
Transforming complex documents into semantically rich embeddings and storing them in Qdrant ensures rapid, accurate information retrieval. This process is integral to the Retrieval Augmented Generation (RAG) approach, enhancing the AI’s output with real-time data retrieval.
Architecture
Figure 1 – AWS reference architecture.
To operationalize the flow, the following process was applied:
- The active customer can upload documents and interact with the system via a web-based user interface, deployed using AWS Fargate – a fully managed serverless container service that can automatically scale up and down based on demand.
- AWS Fargate initiates the process, hosting an API, which orchestrates the entire document and data management sequence.
- AWS StepFunction orchestrates the workflow, starting with invoking AWS Lambda to segment the audit questions into batches which are then processed separately. AWS Lambda executes a LangChain chain, encompassing answer generation, scoring, and reasoning for each question.
- Subsequently, StepFunction consolidates all processed questions into a comprehensive, detailed report, encompassing every inquiry. This is followed by creating a short report tailored for audit purposes, outlining ISO 27001 requirements with compliant or non-compliant statuses for each criterion, via the LLMs.
Building on the solid foundation of a scalable, AI-driven architecture, the journey into refining cybersecurity solutions dives deeper into the critical aspect of LLM evaluation and testing. This phase is instrumental in ensuring that the solution not only performs optimally under diverse conditions but also remains aligned with the dynamic nature of cybersecurity threats and evolving compliance standards.
The evaluation strategy incorporates advanced tools like Ragas and Langfuse, involving a set of rigorously defined performance metrics assessing the LLM output in terms of relevancy, precision, hallucination, and the faithfulness of generated responses.
These metrics serve as a quantitative foundation for continuous improvement, enabling the refinement of models with precision and confidence. Figure 2 below shows one example of the evaluation strategy using Langfuse.
Figure 2: Langfuse interface
The key to success in optimizing LLM performance was the collaborative development of a test dataset in partnership with Peak Defence. This dataset is specifically designed to challenge models, simulating a wide range of cybersecurity scenarios and compliance requirements.
Through targeted experiments and continuous learning mechanisms, performance improvements are tracked over time, ensuring the solutions evolve in lockstep with the latest cybersecurity trends and technologies.
Figure 3: Testing dataset
After collecting feedback, the team systematically processed and analyzed it to distill actionable insights. These insights were then incorporated into the dataset, serving as a foundation for further model fine-tuning and enhancing the AI solution.
This cycle of feedback analysis and dataset integration is crucial for the continuous improvement of the solution, ensuring it evolves in line with user expectations and technological advancements.
Conclusion
The AI automation platform was launched within three months. Now the AI solution can generate compliance reports and responses to security RFPs within minutes compared to the 2-3 weeks of manual effort previously required.
Thanks to automated infrastructure provisioning through GitHub Actions and AWS Cloud Development Kit, the new environment can now be deployed from scratch in as little as 15 minutes. Ultimately, compliance consultancy that once caused bottlenecks due to manual processes can now scale efficiently to serve more customers.
- Previously, the Peak Defence team had to handle an ever-increasing volume of repetitive tasks that drained resources and risked burnout among employees, leaving room for human error.
- Now, having automated some of the manual steps while still applying higher-level human thinking, Peak Defence saves valuable time for its team and ensures they can offer the highest quality for highly sensitive tasks.
- Peak Defence’s AI automation platform can now produce audit reports in a matter of hours, if not minutes, depending on the company’s size.
- This lets customers move through compliance processes much faster without missing important details or overlooking crucial information.
- The platform helps Peak Defence provide compliance consulting to customers much faster, without reducing the quality of their service.
Neurons Lab’s global team includes data scientists, cloud specialists, domain and user design experts, plus business strategists, all supported by an extensive talent pool of 500+ engineers. They solve the most complex AI challenges, including automating business operations in cybersecurity with generative AI. To learn more about Neurons Lab and its services, visit the website and review its offerings in AWS Marketplace. For more information, please contact [email protected].
Customer Feedback
“We needed a professional and flexible partner while going through a fundamental pivot. Neurons Labs helped us to add generative AI capabilities to an existing successful platform. This enabled Peak Defence to evolve and unlock new capabilities for its customers. Neurons Lab’s team brings experience, good organization, and creativity to the projects. That combo makes it efficient and fun to work together. We recommend these guys without hesitation.”
– Roman Jasins, Co-Founder & Board Member of Peak Defence
.
NeuronsLab – AWS Partner Spotlight
NeuronsLab is an AWS Specialization Partner and AI consultancy that provides end-to-end services—from identifying high-impact AI applications to integrating and scaling the technology—that empowers companies to capitalize on AI’s capabilities.
Contact Partner | Partner Overview | AWS Marketplace | Case Studies