How Wix Builds a Platform as a Runtime (PaaR) for Faster, High-Quality Development

Monolith vs. Microservice vs. Serverless

• The question of whether to build a monolith, microservice, or serverless architecture for a next service is complex and depends on the technology and stack chosen, as each has its own strengths and weaknesses (27s).
• Monoliths are efficient and performant because everything runs in-process, while microservices allow for ownership and scalability, and serverless provides easy deployment and scalability (1m29s).
• The growing complexity in modern software systems is a major challenge for software developers, and finding a way to regain control without losing the benefits of different technologies is crucial (1m33s).
• When choosing a technology, developers must trade off between three pillars: how to code, how to deploy code, and how to run or operate code (2m6s).

Wix's Scale and Engineering Challenges

• Arian is the VP of Engineering at Wix, a leading website builder platform that provides a range of solutions, including e-commerce, events, and booking, and allows users to write code and use serverless technology (3m10s).
• Wix has over 250 million website builders using its platform, 7% of the internet's websites run on its platform, and it has a billion human users visiting its websites, with 4,000 microservices clusters across three data centers (4m5s).
• As engineers, it is their responsibility to deliver business value, and successful companies need to provide high-quality code fast to beat their competition (4m49s).

Monolithic Architecture

• Building a monolith can be complicated, as it involves building a single, large application (5m21s).
• A monolithic service is simple to start with, as everything runs in the same process, making life easier for developers, especially for startups, with a single service to manage (5m23s).
• The pros of a monolithic service include easy coding, accessibility, and testing, as well as simple topology, but as it scales, it can lead to mixing domains, spaghetti code, and difficulties in synchronizing between teams (5m45s).

Microservices and Serverless Architectures

• Microservices and serverless systems are more complex, with distributed systems and indirect dependencies, making it harder to manage and test, but allowing for single responsibility and easier deployment (6m26s).
• The trade-offs of microservices and serverless systems include complexity, cross-cutting concerns, and difficulties in refactoring and breaking APIs, but they also offer clear ownership and scalability (7m38s).

The Ideal System and the Task Management Example

• The goal is to find a system that solves most of the issues with code, deploy, and run, and to build something that tackles each pillar and figures out how to build something else (8m39s).
• Dana, a new developer, is tasked with writing a simple task management system, which requires domain modeling, API design, request flow, authentication, authorization, input validation, and object mapping (9m12s).
• The task management system is a simple example, but it still requires consideration of various aspects, such as request flow, authentication, and input validation, making it a complex task (9m35s).
• A developer, Daniel, is tasked with building a simple task management system but faces numerous challenges, including figuring out APIs, database connections, RPC calls, secret fetching, data access, domain events, error handling, GDPR, PII, caching, logging, and testing (10m49s).
• Daniel becomes frustrated with the complexity of the task and the numerous considerations required to complete it (11m27s).

Frameworks and Best Practices

• The solution to Daniel's problem is to use best practices for each of the challenges, which are documented in a framework (12m19s).
• The framework provides recommendations and documentation for using identity, understanding GDPR laws, sending webhooks, and communicating with databases (12m41s).
• Despite the framework's help, Daniel is still overwhelmed, leading to the realization that increasing programmer productivity is not about writing more lines of code but about eliminating unnecessary code (13m39s).

The Journey to Code Faster

• The goal is to eliminate 80% of the code that needs to be written for an app, and the fastest way to write code is to not write it at all (14m2s).
• A journey to code faster in a complex environment began four years ago, involving weekly meetings with the CEO to review thousands of lines of code and identify unnecessary lines (14m29s).
• The focus is on removing lines of code that do not contribute to business logic, as that is what developers are paid to build (15m5s).

Platform as a Runtime (PaaR)

• The solution to removing unnecessary code is to build a platform as a runtime (PaaR) (15m14s).
• A platform called Nile was built to codify guidelines and best practices, allowing developers to work within the platform and avoid cross-cutting concerns, with features such as a framework and search integration (15m17s).
• The Nile platform automates tasks such as indexing and updating documents in Elastic Search, eliminating the need for developers to annotate domain objects (15m55s).
• The platform was built to help with coding and is one of the pillars of the development process, with the goal of making development faster and more efficient (16m25s).

Serverless Platform Development

• In parallel with building the Nile platform, a serverless platform was also developed to provide a customized solution for the company's needs (16m36s).
• The serverless platform was built to address the issue of having a large footprint in production environments, where most of the code running is not written by the company, but rather consists of frameworks and libraries (18m21s).
• The company's software stack typically consists of virtual machines, containers, microservice frameworks, internal frameworks, and business logic, with the goal of building a Pyramid of software development (17m17s).
• Microservices are packaged with frameworks, libraries, and business logic, but most of the code running is not written by the company, which can become a problem when running thousands of microservices (18m19s).
• The life cycle of frameworks and libraries is tied to the product life cycle, making it difficult to update and manage multiple versions in production (19m7s).
• The company faces challenges in updating and patching common libraries and frameworks, which can be a long and complicated process, especially when dealing with thousands of microservices (19m38s).
• Many companies have services without owners, and supporting multiple languages can be an issue, requiring duplication of platforms and frameworks, which is costly and time-consuming (20m10s).

The Ideal Platform and Wix's Approach

• The ideal platform would have easy and fast code, minimal integration tests, and no boilerplate, with fast deployment, scalability, and low cost (20m49s).
• Every system and stack has pros and cons, and the goal is to take the best of microservices, serverless, monoliths, and managed platforms (21m27s).
• Wix is a managed platform for users, allowing them to write code on top of their website, with Wix handling deployment, Kubernetes, and database provisioning (21m45s).

First Attempt at PaaR with Node.js

• The concept of "Platform as a Runtime" (PaaR) was developed to address these issues, with the goal of building a platform that can run multiple languages and frameworks (21m55s).
• The first attempt at PaaR used Node.js, with an application framework, service integration layer, and data services layer, allowing users to build their business logic on top (23m3s).
• Node.js was chosen for its lightweight nature, dynamic code loading, and ease of learning, and the application framework handles HTTP headers, authentication, and monitoring (23m39s).
• The service integration layer provides RPC clients and libraries, allowing users to integrate their applications without doing lookups or integration (24m5s).
• The data services layer uses DynamoDB as a key-value store, and the platform is packaged and put on the cloud, with user code deployed on top (24m33s).
• Code can be loaded dynamically into a platform without the need for the entire platform, just the interfaces, allowing for a trusted environment where multiple functions or small services can be added to the same container (25m5s).
• This approach differs from AWS Lambda, which is a non-trusted environment requiring everything to be packaged together, making it impossible to share the platform between different services (25m19s).
• The platform has no integrations as they are provided by the platform itself, resulting in less testing, faster deployment, and zero boilerplate code (26m16s).
• Developers can focus on small functions without the overhead of large packages and frameworks, making deployment very fast (26m27s).

Improved Developer Experience

• The developer experience is improved by taking concepts from a previous project, Nile, and re-evaluating every line of code to determine if it should be provided by the platform (26m49s).
• A developer, Dana, has a change request to add an API to retrieve task details and assigned persons, and write an audit log to a database, which can be done as a separate service in a serverless function world (27m9s).
• Dana only needs to write a small amount of code to import the task server, expose a new API endpoint, call the task server, extract the contact ID, call the contact server, and return the task and contact details (28m35s).
• Adding a Kafka consumer is also simplified, as Dana can get the Kafka consumer and data source from the context without writing any connection handling code (29m20s).
• A developer, Dana, can write code without having to worry about boilerplate code, and once she pushes the code, it's running on production within one to two minutes, thanks to the platform's minimal code and small deployable size (29m52s).

Optimizing the Run Pillar

• The deployment pillar is built as a platform as a service, and the concept is being applied to the Run pillar to optimize it (30m32s).
• The Run pillar uses a serverless node runtime, allowing for ownership and control, even with a serverless strategy, by giving containers to each team or business unit (31m3s).
• The platform can handle functions running in-process and dynamically loaded into the platform, making it easy for developers like Dana to write code without worrying about the underlying infrastructure (31m57s).
• The platform can scale by putting functions on another container and load balancing between them, and it can also optimize function affinity by deploying frequently called functions in-process (32m19s).
• The platform can optimize the function affinity by deploying frequently called functions in-process, replacing network calls with in-process calls, and reducing latency (32m43s).
• The platform's deployment strategy allows for small functions to be deployed without the framework, a single version of frameworks and libraries, and decouples the life cycle of frameworks and libraries from the life cycle of products (33m28s).
• The platform's deployment strategy also allows for easy deployment of the platform, without requiring a cross-company effort, and enables all teams to get the updated platform at once (33m55s).

Adding Multi-Language Support

• The platform is missing the ability to add an additional language, which is being worked on in version two, to support languages other than TypeScript, such as Scala (34m27s).
• A lot of progress has been made on the platform, with thousands of functions running for two years, and developers love it, with the next step being the Wix single runtime (34m51s).

Wix Single Runtime

• The Wix single runtime involves tradeoffs, including switching from in-process to out-of-process calls, with the product teams building business logic using an SDK that communicates with the host platform (35m11s).
• The architecture consists of a demon set with one container for incoming and outgoing calls, and every function or microservice runs on its own pod on the same machine, with local host communication being faster than network communication (35m56s).
• The tradeoff of using out-of-process calls is offset by gaining the Kubernetes ecosystem for deployment and autoscaling, which would have had to be reinvented otherwise (36m51s).
• The solution has a larger footprint than the previous TypeScript solution but is still 50% lower than packaging the entire framework inside a microservice, with a JVM footprint (37m12s).
• Benchmarks showed a performance loss of about 2 milliseconds compared to an in-process microservice, which is tolerable, and it's believed that this loss can be gained back in a distributed system (37m39s).
• The deployment strategy involves local host communication, which can potentially offset the 2-millisecond overhead, and performance tests showed that up to 15,000 RPMs, the overhead is about 2 milliseconds (38m40s).
• Services with millions of RPMs may not be suitable for this solution and can be packaged as standalone microservices instead (39m3s).
• The work setup has been changed to allow packaging the platform inside a regular microservice with just a flag on the build system, and developers still don't need to change their code (39m26s).
• A high-scale system can be packaged together with a standard framework, allowing for relatively low RPMs, and can be used for systems with up to 30k-50k RPMs, which is still a lot, and provides cost savings and a single framework for fast deployment of security changes or legal constraints (39m41s).

Benefits of the Platform

• The platform takes care of GDPR for users and supports multiple languages because it switches from an in-process to an auto-process, requiring only the investment of writing an SDK to support a different language, which is much cheaper than keeping feature parity between frameworks (40m17s).
• The platform is coded, deployed as a service, and run as a bar or as a virtual monolith, taking the best of all worlds and creating a platform as a runtime, allowing for business value to be brought fast (40m55s).
• The improved velocity of developers increased by 50 to 80% due to writing less code, and costs on compute can be reduced by about 50% because more density can be pushed into a single node, with the footprint of a single service being about 50% smaller (41m31s).
• The platform allows for faster deployment of security changes or legal constraints, such as GDPR, and provides support for multiple languages, making it a cost-effective solution (40m11s).

Backward Compatibility and Platform Evolution

• When a breaking change occurs, backward compatibility is kept, and proxies are created to change the previous API to the new API, ensuring a smooth transition (43m41s).
• The platform's framework is designed to keep up with architectural requirements, and changes can be made to accommodate different types of requirements, such as dedicated different types of services (44m25s).

Opinionated System and Developer Freedom

• A highly opinionated system is necessary for scaling, which means limiting the freedoms of developers to ensure consistency across different stacks of technologies (44m46s).
• The platform provides a set of tools and technologies that developers are expected to use, similar to how AWS or Google Cloud provide their own set of tools and technologies (45m7s).
• While the platform provides a lot of value to developers, there are cases where developers need to opt out of the platform, and in such cases, they should first try to solve their issues within the platform (45m34s).
• The platform is built with layers, allowing developers to take the lower levels of the platform and use the same core libraries, but without the synthetic sugars and automations (45m41s).
• In rare cases, developers may need to use a different technology stack, such as the media platform at Wix, which requires video encoding and image manipulations and uses a different platform (46m1s).
• For most cases (80-90%), developers should stick with the platform, as it provides a lot of value and makes development easier (46m41s).
• Developers tend to try to stay within the platform and avoid opting out because they realize the value it provides (46m54s).

Testing and Local Runtime

• The concept of using a boring architecture is similar to the idea of using a platform, as it provides a set of proven and reliable technologies (47m6s).
• Testing functions and APIs can be done without having the whole environment, and Wix provides a local runtime for testing, as well as the option to test on production using a test tenant (48m3s).
• Wix's multi-tenant system allows developers to create their own test tenant and test end-to-end flows without corrupting other tenants (48m30s).
• The deploy preview feature, part of Wix's CD/CI system, allows developers to deploy and test their code in a production-like environment (48m43s).
• In a preview GA, a specific artifact is deployed but doesn't receive any traffic, unless a test request is made with a special header, which routes the test calls to the new deploy artifact, allowing it to interact with other artifacts and APIs without corrupting data (48m52s).
• The platform prevents users from making tenancy mistakes by injecting the tenant ID into queries, which are not allowed to be tampered with, and the platform injects the tenant ID from the authentication headers (49m27s).
• The system is designed to be fairly safe, preventing users from corrupting any data, thanks to the platform's handling of tenancy and injection of tenant IDs into queries (49m44s).