AI at Ollion: Practical Solutions, Real Value
In today's fast-paced technological landscape, artificial intelligence (AI) has become an indispensable asset. At Ollion, our commitment is to fully leverage AI by ensuring its accessibility throughout our organization. Our approach to integrating AI into our internal platform has been guided by a dual focus on usability and responsibility. We have successfully embedded AI into our operations, with a strong emphasis on safe usage, cost-effectiveness, and insightful usage analytics. This case study details our strategic methodology and the innovative solutions employed to cultivate a progressive AI ecosystem within Ollion.
The Platform
Core to Ollion’s managed service offering, our internal platform is built entirely on AWS-native services. It features a dynamic, event-driven architecture composed of independent microservices, readily accessible and integrable via APIs. Our robust system supports automation and workflow needs across the organization, focusing on:
Operational Monitoring
Providing automated alert configurations, data ingestion, processing, and remediation to ensure comprehensive operational oversight.
Security and Governance Monitoring
Leveraging native service configurations to enable thorough findings collection, aggregation, and analysis.
Cost Optimization
Enhancing cost management by transforming, analyzing, and visualizing billing data.
Billing Management
Supporting billing (CUR) file ingestion, processing, and invoice generation.
Centralized Reporting
Facilitating report generation, scheduling, and delivery.
Data Access and Control
Providing access to internal data assets while ensuring proper security and compliance control.
Adding AI to the Mix
In developing the AI component of the platform, we established the following criteria to guide its development:
API Accessibility
Ensure all features are accessible via APIs to facilitate integration and usability.
Token-Based Authentication
Implement token-based access to manage authentication and authorization for specific features.
Logging & Analysis
Enable logging and analysis to monitor engagement and ensure safe usage practices.
Cost Control
Establish mechanisms for monitoring and controlling costs.
Documentation
Provide documentation outlining each API endpoint and its functionality to assist developers.
CI/CD Pipelines
Utilize CI/CD pipelines triggering IAC (CDK) for seamless deployments and updates.
Flexibility Deployment
Provide for easy deployment in any AWS environment to enhance scalability and adaptability.
Although the AI component could have been integrated as a direct add-on to the existing platform, we decided to develop it as a standalone system. This isolated approach offers tighter control over access, compliance and costs, while also allowing us to create a distinct ecosystem that can be replicated and adapted for use in other parts of the organization. Additionally, the AI infrastructure mirrors the architectural patterns used in the rest of the platform. APIs are hosted on ECS with deployments powered by CI/CD pipelines hosted in CodePipeline. All underlying infrastructure components such as VPCs, system parameters, secrets, S3 buckets, DynamoDB tables and Postgres PG vector instances are deployed using CDK. This strategy allows us to maintain flexibility and consistency across our technological ecosystem, enabling scalability and easy adaptation within the organization.
Platform AI Architecture
Integrating Amazon Bedrock
Our platform APIs are designed to streamline and simplify interactions between applications and Amazon Bedrock, emphasizing the features most frequently used for AI solution development. By focusing on these core capabilities, we ensure that developers can harness the power of Amazon Bedrock without being overwhelmed by its full spectrum of functionalities. Key capabilities of our platform AI include:
- Model Management: List available models and provide detailed information for each model
- Document Processing: Load and parse documents for embedding, as well as chunk documents and create embeddings for storage in vector databases.
- Conversational Functionality:
- Provide both standard and streaming responses
- Converse using a text prompt only
- Converse using a text prompt and an attached image
- Converse using a text prompt and an attached document
- Agent Commands: Work with the fundamental Bedrock agent commands.
By supporting only critical parameters in API interactions, our platform ensures ease of use and rapid adoption. For users needing more advanced features, the system's architecture allows easy scalability, customization, or full replication in different environments for tailored applications.
Providing visibility into AI Operations
Visibility into AI endpoint usage, compliance, and costs is a focal part of our approach. To achieve this, every API call—including any errors encountered—is logged in Amazon S3. These logs capture original messages, uploaded files, and responses. Storing logs in S3 allows for efficient querying using Amazon Athena, visualization with Amazon QuickSight, and further analysis through Amazon SageMaker. Cost monitoring is facilitated by the platform's billing file ingestion processes and is visualized in the platform portal's cost optimization dashboards, providing comprehensive insights into expenditures.
Example Cost Dashboard
Putting It All Together
By integrating Amazon Bedrock features through a set of streamlined APIs, AI has transitioned from a "nice to have" to an essential component of our operational foundation. This ensures that developers have the necessary tools to seamlessly create and deploy AI-driven applications, significantly enhancing our capabilities. Here's how Ollion leverages platform AI to promote innovation and efficiency:
Internal Documentation Search
Documents from external sources are chunked, embedded and loaded into a vector database to facilitate natural language searches
Natual-Language Data Queries
General inquiries are automatically transformed into SQL queries, which retrieve data and provide summaries.
Invoice File Parsing
Received invoice files are parsed and transformed into JSON for streamlined data storage and processing.
Visualization Generation
Raw infrastructure metrics are collected and sent to AI for visualization generation.
Cost Analysis Reports
Portal cost views are converted into deliverable reports featuring visualizations, analysis, and summaries created by AI
Alert Ingestion analysis
Incoming infrastructure alerts are analyzed for anomalies, correlations, and more.
Support Tickets Enhancement
Support tickets are enriched through automatic documentation searches and summarization improving response quality.
Making AI Make Sense From Innovation to Execution
At Ollion, we are dedicated to the practical application of AI technologies to enhance efficiency and deliver business value. Our goal is to embed AI into our platform not just for innovation, but to optimize operational processes and maximize resource utilization. This ensures technology acts as a key enabler for organizational growth and success. As we further develop our platform, we will advance our AI adoption to accelerate service delivery, particularly in responding to and resolving client requests and escalations more quickly. Join us in integrating these AI-driven solutions, which are designed to meet the real-world needs of our teams and business, and establish a foundation for sustained achievement in the AI era.
