News

Atlassian Puts Local AWS Stack on GitHub for Testing/Mocking

• By David Ramel
• March 27, 2017

For all the developer goodies on the Amazon Web Services Inc. (AWS) cloud, some coders can find it hard to create, manage, synchronize and test development and production processes with accuracy and reliability.

To provide a better handle for some of those processes, enterprise software company Atlassian has just put a project on the GitHub open source code repository called LocalStack, a "fully functional local AWS cloud stack" for testing and mocking cloud development projects.

While it may presumably be able to work with other cloud providers in the future, the tool now targets only the AWS platform, letting developers create and test cloud apps while offline.

Although LocalStack isn't new -- having been published on the Bitbucket hosting service in August 2016 -- its placement on GitHub has generated some buzz in the developer community.

"One of the biggest 'drawbacks' of using AWS as a production platform is that making your development environment look like production is hard," reads one post in Hacker News comments. "Having to deploy to test is cumbersome and having a cost associated with each test can definitely introduce some sort of 'stress' and encourage people to not test incrementally. I wonder if this changes that. Having services like S3, Lambda and SQS available locally sounds super interesting."

In addition to S3, Lambda and SQS, developers using LocalStack on their local machines can also spin up core cloud APIs for a bunch of other AWS services, including API Gateway, Kinesis, DynamoDB, DynamoDB Streams, Elasticsearch, Firehose, SNS and Redshift.

"Additionally, LocalStack provides a powerful set of tools to interact with the cloud services, including a fully featured KCL Kinesis client with Python binding, simple setup/teardown integration for nosetests, as well as an Environment abstraction that allows to easily switch between local and remote Cloud execution," the project site says.

Atlassian, perhaps best known for its JIRA issue tracking tool, said LocalStack combines and builds upon existing "best-of-breed" mocking/testing tools: primarily kinesalite/dynalite and moto.

Those tools lack functionality for some use cases, the project description states, so Atlassian combined them, made them interoperable and added missing functionality that it describes thusly:

• Error injection: LocalStack allows developers to inject errors frequently occurring in real cloud environments, for instance ProvisionedThroughputExceededException which is thrown by Kinesis or DynamoDB if the amount of read/write throughput is exceeded.
• Actual HTTP REST services: All services in LocalStack allow actual HTTP connections on a TCP port. In contrast, moto uses boto client proxies that are injected into all methods annotated with @mock_sqs. These client proxies do not perform an actual REST call, but rather call a local mock service method that lives in the same process as the test code.
• Language agnostic: Although LocalStack is written in Python, it works well with arbitrary programming languages and environments, due to the fact that we are using the actual REST APIs via HTTP.
• Isolated processes: All services in LocalStack run in separate processes. The overhead of additional processes is negligible, and the entire stack can easily be executed on any developer machine and CI server. In moto, components are often hard-wired in RAM (for example, when forwarding a message on an SNS topic to an SQS queue, the queue endpoint is looked up in a local hash map). In contrast, LocalStack services live in isolation (separate processes available via HTTP), which fosters true decoupling and more closely resembles the real cloud environment.
• Pluggable services: All services in LocalStack are easily pluggable (and replaceable), due to the fact that we are using isolated processes for each service. This allows us to keep the framework up-to-date and select best-of-breed mocks for each individual service (for example, kinesalite is much more advanced than its moto counterpart).

Prerequisites to use the tool include: the "make" build automation tool; Python; the pip Python package manager; the npm package manager for Node.js; and the Java runtime environment and compiler (java/javac). Developers can also use LocalStack in a Docker container, obviating the need for installation requirements.

The project comes with a set of prebuilt unit and integration tests, along with a Web dashboard that helps keep track of AWS components and the relationships among them.

As of press time, the project had more than 3,000 stars, with the total climbing fast.