An Introduction for Developers

Service virtualization is increasingly popular in the software & DevOps toolkit. Learn what it is, how it works, and how to use it.

Service virtualization is an increasingly popular tool in the software developer and DevOps toolkit. This blog covers what it is, its use cases, and how it works. This introduction is part of our in-depth series on service virtualization. Stay tuned and follow along for more content!

What is Service Virtualization?

The Virtual Service Concept

Most web applications are connected to several dependent services (unless perhaps it is a system of record). For example, an eCommerce site may be linked to payment, inventory, shipping, and logistics software. This interconnectedness has made software testing challenging. Conventional methods at many engineering organizations often leverage a replica of an end-to-end production environment, but that soon proves too complex to manage.

Enter service virtualization.

Service virtualization simulates backend service dependencies in a service-oriented or microservice-based architecture. Each simulation is typically referred to as a virtual service.

By creating virtual services that represent unavailable APIs, costly backend systems, or services without the right data, developers can minimize delays and get around their current development and testing constraints.

The Origin of Service Virtualization

Service virtualization is not a new concept. In fact, it’s been around since 2002, when it was first popularized by a startup called iTKO. iTKO was responsible for “productizing” service mocking for the masses. Virtual services from iTKO were used to stand in for APIs, message queues, mainframes, and databases.

Since then, several new players have emerged in the space. The core concept of service virtualization is a solution to the common question of how one should test an application in the software development lifecycle. This is a bit of a “chicken or the egg” question – how do you test a service for faults when you can’t introduce those faults into the service for practical reasons?

Utilizing a virtual environment and virtualized services bypasses this question altogether, offering a test base of systems without directly impacting the production service.

Case Study: Nylas

Using 3rd party mocks, Nylas improved performance by 20x with Speedscale.

Use Cases for Service Virtualization

Simulating Unavailable Services

Service virtualization can simulate an API response from an unavailable service. There are several reasons why a service may be unavailable:

• The service is not built or updated yet, even if the service structure is already planned or partially in place.
• The version in the testing environment has drifted from the version in production, creating a misalignment between the stated endpoints or functionality and the integrative pathways to use them.
• It is a 3rd party service, which often means highly removed observability of the internal pathways and functions.
• There is only one instance of it (e.g., it is a pay-for-use sandbox), and it is thus unavailable for continuous testing.
• The system of record, which is the backend of the service, has incorrect data, and therefore, you can’t find the actual service to connect with

Inadequate Performance of the Real Services

When the real API doesn’t perform as you need it to, service virtualization can step in. For example, if the external API responds with high latency, using virtual services would allow you to speed up or slow down response times.

Or perhaps the real service can only scale to a particular TPS (transactions per second) below the benchmark you are trying to achieve. For example, a vendor-provided sandbox API for a payment gateway only reaches 10 TPS max. Still, your manager wants to see if your order processing API can go to 2,000 TPS in preparation for the holidays. Service virtualization would allow you to increase that limit.

The Responses of the Real Backend Do Not Meet Developer Needs

Development teams may feel stuck if they need to execute certain test cases but cannot get a specific response from the actual service. For example:

• Developers sometimes need to solicit additional error codes from the backend, but in practice, they can only access specific responses or generate specific errors.
• The API test data only contains a few records, but you need several thousand to effectively process the scope and range of the error.
• The actual backend (such as a third party) has a wide variety of responses that get triggered, but you only need a select few – this can introduce a ton of noise that makes specific testing difficult and highly expensive.

These scenarios can be easily programmed into a virtual service.

Other Service Virtualization Examples

Service virtualization is a common practice across many industries and organizations for different functional and performance testing purposes. Consider these examples:

• Gig, rideshare, and delivery companies such as DoorDash, Instacart, and Uber must simulate mapping APIs to pretend their drivers are en route.
• Retailers must simulate a payment gateway, inventory, or shipping system to test the payment and provision flow against the design spec.
• SaaS software providers must simulate authentication and user lookup APIs to see whether their data flows are secure and their data provision effective.
• FinTechs must simulate online banking APIs, low-latency trade platforms, or credit checks.

Feature Spotlight: Service Mocks

Service Mocks–another name for Service Virtualization–are a critical component of development and testing. Learn the how & why behind our Kubernetes traffic-based mocks.

How Does Service Virtualization Work?

Recording Traffic

Virtual services are created by listening to traffic. Almost all service virtualization tools require a proxy to be inserted between two services, which requires temporary configuration or code changes.

More modern service virtualization solutions, such as Speedscale, allow transparent proxy listeners like Kubernetes sidecars to be installed, requiring no changes for recording or playback.

Editing the Service

After a service is recorded, most tools require editing the service images or data files. A UI editor or scripting format is normally available, and creating service response logic can get complex. Be aware that when statefulness and data idempotence are required, services may take weeks to complete.

Replaying the Service

Virtual services or service mocks are typically deployed on a server and require the client API to be redirected to interact with the virtual endpoint. When development or testing is complete, the changes must be reverted. Modern solutions such as Speedscale allow containerized virtual services to be deployed in any cloud/cluster with unlimited instances.

Why Do You Need Service Virtualization?

Service virtualization is a critical tool for solving the intrinsic problems of testing a production system. Testing the actual services for faults or issues means invoking those problems, and as such, it has a necessary impact on the production services themselves—by virtualizing those systems, you abstract the physical server or the actual service into a virtualization tool and create an option that does not directly touch the production system.

This allows for a substantial leveling up for your testing process. API virtualization allows for comprehensive testing against a system as close to production as possible without introducing the chaos or damage that actual testing might incur. This unlocks substantial benefits for continuous testing, load testing, integration testing, and other security testing solutions.

Mocking vs. Service Virtualization

Engineers have been using stubs and mocks in the development process for years. While mocking involves testing specific software components with limited context, service virtualization is essentially a more sophisticated version. Service virtualization replicates systems at the production scale for more robust testing. Generally, mocks are manually developed for each test case.

Engineers may use the terms service mocks, test doubles, and virtual services interchangeably – so just be aware. For a more stringent definition, you can consider mocks to be a more fundamental building block for what service virtualization is:

• Mocks: a specific component is abstracted for testing, creating a particular and isolated element for testing.
• Service Virtualization: Many mocks and their supportive systems are combined to create a cohesive and complete environment for testing.

Benefits of Service Virtualization

Release can be delayed or even halted when services in end-to-end environments are unavailable. However, leveraging virtual services can circumvent problems in a timely manner. Now more than ever, preserving developer time is of growing importance in an age where highly paid engineers bear multiple responsibilities. Ideally, they release customer-facing features that generate revenue and differentiate the enterprise.

Some specific benefits of virtual services include:

• Reduced reliance on complex end-to-end environments, reducing dependent systems, and making more straightforward web services.
• Increased availability of development environments, directly benefiting your development and testing teams.
• Broader data responses for more robust unit, functional, integration, and load testing; load tests in particular, benefit from this virtualization as you can scale up and down more freely.
• Faster release cycles, since any unavailable systems can be simulated, this has long-term benefits to the software development life cycle.
• Simulated what-if simulation capabilities to ensure cloud resiliency and scaling under high volume and a variety of other different scenarios.
• Peace of mind and confidence in releases.

How Speedscale Can Help With Service Virtualization

Speedscale can help with service virtualization in several ways. Speedscale can automatically generate sophisticated mocks of third-party services and internal APIs based on actual traffic patterns. This allows developers to simulate critical application integrations without complex, end-to-end staging environments.

By capturing and replaying actual user behavior and traffic, Speedscale can create realistic local testing environments, enabling engineers to provision API, database, and message queue mocks within minutes.

Speedscale’s API virtualization enables resiliency testing by creating a test environment that can be used to test specific, often chaotic issues. For instance, this virtualization makes it possible to simulate issues such as latency or service 404 errors, allowing for more control over otherwise chaotic testing conditions.

Speedscale is specifically designed to work with containerized solutions such as Kubernetes or Docker. It allows mocks to run as pods alongside applications, serving as ephemeral preview environments that utilize actual data. This integration ensures the simulated services closely mirror production conditions, leading to more accurate and useful testing outcomes. You can also port this data out to your solution if you don’t use containers or don’t want to engage in testing in this way, giving you incredible flexibility in your testing approach.

Conclusion

There are many additional concepts, which we will explore in subsequent blogs, but virtual services can be a powerful accelerator for software development teams looking for more velocity. Some service virtualization tools on the market even offer more advanced features to take your software testing to the next level:

• Parameterized data values to augment fields with datasets, allowing you deeper visibility and permutability within the data state
• Self-healing virtual services to keep pace with contract or schema changes, allowing you to rapidly iterate the service without incurring much of the expensive change management that comes along with it
• Pass-through modes to avoid config/code changes, allowing you to bypass changes, engage in complex testing, or provide ongoing functionality even in a shifting environment
• Autoscaling virtual services to meet demand, allowing you flexibility that significantly aligns with business and service objectives (especially when you have an SLA in place)
• Adjustable latency, allowing for more complex network simulation and emulation
• Chaos responses – introducing chaos in a controlled sense

We’ll cover top tools in an upcoming post – stay tuned!