Ben Markines and Nicolas Espejo are Principal Engineers and Infrastructure Managers at Split. They were tasked with evolving the platform safely without impacting customers. Today, they’re sharing their knowledge on building, deploying, and managing agnostic artifacts.
At Split, our application used to run on a single cloud provider. As we mature and expand, new demands shape the evolution of the application. Customers require a disaster recovery plan and want the application to run across multiple continents to provide the best user experience. Therefore, the Split application must satisfy local laws and regulations. Beyond just running in a single region, we’ve been questioning if removing a single cloud provider lock-in is necessary.
The first realization we had was that our software needed to be smart enough to determine its own execution environment. Then, we requested the necessary configurations to ensure proper operation, so an agnostic artifact becomes a hard requirement and continues growing. Just as we had our initial realization, we also faced our first challenge, which involved developing a library with necessary intelligence and adapting all of our services accordingly.
Migrating the Messaging Stream Solutions
Today, Split leverages many cloud-provided solutions. One service is the messaging stream solution. We accomplish its safe migration to Kafka for our event streaming requirements by leveraging feature flags. With Split’s feature flagging platform, teams have control to roll back in case the application does not perform to expectations. Furthermore, the rollout plan involves control of which customers leverage the messaging solution for tuning and scaling. All these controls and benefits are made possible by Split. In addition to our migration to Kafka, we also migrate away from cloud-specific storage, database, and data pipeline processing solutions. Split allows us to migrate off of vendor solutions to a generic solution.
But it wasn’t just about software. We also had to revamp our entire infrastructure using Kubernetes and embrace Infrastructure as Code (IaC). This was a big deal because it allowed us to keep things consistent across environments and regions, and also to accelerate our replication. A secret/parameter manager solution played a crucial role. The adoption of this technology allowed us to achieve region/environment independence for our artifacts and became an essential dependency for all our services.
Kubernetes enables Split to manage and scale across different cloud providers and regions, ensuring a consistent and reliable user experience, internally for our developers and externally for our customers. Artifact-agnostic software allows Split to provide a global service with high availability and performance while avoiding vendor lock-in and maintaining flexibility to adapt to changing business needs.
Abstractions and Flexibility
Developing artifact-agnostic software is a best practice in modern software development that allows applications to deploy across different cloud providers. As we grew at Split, decisions based on speed and time to market took priority. But as customers required us to run in various regions and different clouds, we realized that some of the decisions previously made needed abstraction. Configurations are abstracted, migration away from cloud-specific services, and leveraging infrastructure as code.
Split needs to deploy on different clouds and various regions within a cloud in order to achieve this, our engineering team adopts an agnostic mindset, designing and developing new features with this flexibility in mind.
Advantages of Artifact-Agnostic Software
- Portability: Artifact-agnostic software can move between cloud providers without modifications to application code. Split takes advantage of value and performance from different cloud providers without being locked into a single provider.
- Consistency: Artifact-agnostic software ensures that the application environment is consistent across different cloud providers, which helps to avoid issues related to environment-specific differences. This consistency allows companies to focus on developing and delivering their applications rather than managing the infrastructure.
- Flexibility: By being able to deploy applications across different cloud provider regions, companies have greater flexibility in how they structure their infrastructure. They can use distinct cloud regions to host parts of their application based on the specific needs of each component.
Disadvantages of Artifact-Agnostic Software
- Performance: While artifact-agnostic software offers portability and consistency, it may not always be the best performance. Cloud providers offer better performance or lower latency than other providers, depending on the specific use case.
- Complexity: Developing and deploying artifact-agnostic software can be more complex and require additional resources to deploy applications to a single cloud provider. Companies need to manage containerization and container orchestration tools, which can add to the complexity of the application deployment process.
- Cost: Running artifact-agnostic software can be expensive, as companies pay for additional infrastructure, network, and maintenance costs. Companies must carefully consider the cost-benefit analysis before deciding to use artifact-agnostic software.
- Developer Time: Instead of building specific features or enhancing the application, developers need to have a broad understanding of the system to make the necessary changes. Developers must balance the age-old problem of tech debt resolution versus new features. Building artifact-agnostic software will certainly trigger these sorts of decisions.
As part of service migration to a more generic posture, Split is a critical tool for safe migration without downtime. Artifact-agnostic software offers portability, consistency, and flexibility. There are also disadvantages, including performance, complexity, and cost. Companies need to carefully consider the trade-offs before deciding to develop artifact-agnostic software. Ultimately, the decision to strictly develop artifact-agnostic software will depend on the needs and requirements of customers.