Spec-Driven Development at Aryeo

Published in Backend and Specs on Apr 29, 2021 by Jarrod Parkes

Zero to Launch

When I joined Aryeo in late 2020, the mission was clear: build Aryeo's first mobile app. After a short discussion with key stakeholders, the engineering team collectively pitched a simple, minimum viable app. The app would expose a subset of Aryeo's functionality with a focus on real-estate agents and brokerages. Not too hard, right? Well, sort of. There was a catch! Aryeo did not yet have an API, so the would-be app had nothing to consume. Fast forward to today, and we have a fully functioning API and new mobile app called Aryeo Listings. This post outlines how we built the Aryeo API using spec-driven development.

What is Spec-Driven Development?

For many APIs, implementation happens in tandem with burgeoning user requirements. Backend engineers create endpoints as the requirements dictate, but otherwise, there is nothing guiding the development of the API. This can be a workable solution for a short period of time, but quickly an API can spiral out of control. APIs built this way often suffer from code duplication, inconsistent data formats, drifting expectations, and a terrible developer experience for downstream frontend engineers. These problems are compounded when dealing with mobile apps because of the likelihood of mobile users running different versions of an app which all make separate assumptions about the current state of the API.

Spec-driven development takes a different approach. Instead of ad-hoc API implementations, each aspect of the API is carefully planned ahead of implementation. Before an engineer writes a single line of code, the details of the API are captured and described in a specification, or "spec" — and sometimes multiple specs. These specs then serve as the de-facto source of truth for backend and frontend engineers as they start their work.

Specs can be constructed in a number of different ways, but for our purposes, we chose to build our spec using Stoplight Studio and the Open API Specification V3. Now for the first step.

Sketching Flows

Like many great ideas, we started our spec-driven process by turning to pencil and paper. Using the same rehashed conversations from our mobile app planning, we sketched a handful of user flows that we foresaw in the mobile app. These sketches formed part of the mobile app's project pitch (à la Shape Up), and were extremely useful in synthesizing the endpoints and data needed from the eventual API.

One key to this step is maintaining proper fidelity in the sketches. You don't need to be an artist or use an extreme level of detail for this step. Instead, just focus on the big ideas and data abstractions. Later, once pieces of the API are implemented, your pencil sketches can give way to more permanent ink.

Sketch to Spec

Going from sketch to spec is an exercise in mapping visuals to requests, responses, and endpoints. For example, one of our sketches referenced a screen showing the details of a real-estate listing. That screen needed to include the listing’s address and sections for uploaded images, videos, and other media. From these elements, we imagined a response payload for a listing:

{
	"id": {integer},
	"address": {
		"line_1": {string},
		"line_2": {string},
	},
	"images": [
		{
			"id": {integer},
			"thumbnail_url": {string}
		}
	],
	"videos": [
		{
			"id": {integer}
			"thumbnail_url": {string}
		}
	]
}

This JSON-like structure could then be further broken into segments, for each JSON sub-object. Using Stoplight Studio, the primary object and each sub-object mapped nicely to model specs:




In a similar fashion, we imagined an endpoint responding with the details about a listing. To get this data, a GET endpoint like /listings/{id} could exist. And again, with the power of Stoplight Studio, specifying the endpoint and response objects is a cinch:

As we reviewed each sketch, we are able to map out nearly a dozen models and endpoints for the API. This included POST and DELETE endpoints too, but we won’t show those examples for brevity. However, if you'd like a more robust look at how to create endpoints and models using specs, we highly recommend checking out Spotlight’s documentation on starting a new API design.

Spec Tools

At this point, we had scaffolded a spec, but it was only as useful as that aging documentation in your company’s internal wiki — you know what I’m talking about! So, how did we avoid this becoming a fruitless endeavor? I’m glad you asked. We introduced two key tools to validate and verify the spec and its eventual concrete implementation.

Spectral

Buried within Stoplight Studio, you may notice a tool called Spectral. Spectral is a powerful JS-based linting tool that can be used to enforce standards across all of your specs — whether they exist in Stoplight Studio or not. For our specs, we created a custom ruleset by extending the standard ruleset defined for Open API Specification V3 (oas3). Here are a few of our custom rules:

  • All models must provide a description
  • All model properties must provide a type, description, and example
  • All string values must defined a minimum and maximum length
  • All property values must be lowercase and snake-cased
  • All date properties must be specified in ISO 8601 format

When editing model or API specs in Stoplight Studio, you get automatic feedback about any violated rules:

You can also run Spectral from the command-line — something we utilize heavily as part of our continuous integration and deployment (CI/CD) process. More on this later.

$> spectral lint models/Video.v1.json 
JSON Schema (loose) detected

models/Video.v1.json
  7:10  warning  model-properties-snake-case       Properties should be lower snake_case.                                  properties.ID
 12:13  warning  model-properties-snake-case       Properties should be lower snake_case.                                  properties.TITLE
 19:21  warning  model-properties-description      Property `type` and `description` must be present and non-empty.        properties.THUMBNAIL_URL
 19:21  warning  model-properties-snake-case       Properties should be lower snake_case.                                  properties.THUMBNAIL_URL
 26:20  warning  model-properties-description      Property `type` and `description` must be present and non-empty.        properties.PLAYBACK_URL
 26:20  warning  model-properties-snake-case       Properties should be lower snake_case.                                  properties.PLAYBACK_URL
 26:20  warning  model-properties-strings-min-max  String properties must specify a `minLength` (0 or 1) and `maxLength`.  properties.PLAYBACK_URL

✖ 7 problems (0 errors, 7 warnings, 0 infos, 0 hints)

Spectator

The second tool, and perhaps the most important for enforcing spec-driven development, is Spectator. This nifty PHP package consumes an API spec and allows you to write tests against it. Since our backend is implemented with Laravel, this package gives us the ability to ensure all our Laravel controllers and resources are written against the spec. The package’s README summarizes this well:

Write [contract] tests that verify your API spec doesn't drift from your implementation…While functional tests are ensuring that your request validation, controller behavior, events, responses, etc. all behave the way you expect when people interact with your API, contract tests are ensuring that requests and responses are spec-compliant, and that's it.

Hot Tip

If you’d like to adopt a tool like Spectator, one difficulty you may encounter is the need to use a single spec file instead of many spec files. If you’ve been building separate model and API specs in Stoplight Studio, then this will apply to you. To merge all your specs into a single spec, Stoplight Studio provides a de-referenced export feature for your API spec. This causes all model references (refs), like the ones used to define an endpoint’s response or request body, to be replaced with the entire contents of the corresponding model spec. If you’re not using Stoplight Studio, don’t worry, similar functionality is provided by command-line tools like swagger-cli.

Once you have a fully de-referenced spec, then you can begin using Spectator to write contract tests. Here’s an example of one of the tests we wrote for our GET /listings/{id} endpoint:

class ApiListingControllerTest extends TestCase
{
    use RefreshDatabase;

    public function setUp(): void
    {
        parent::setUp();

        Spectator::using('Aryeo.v1.json');
    }

    public function testGetListingResponse()
    {
        // listing object is stubbed
        $this
            ->getJson(route('api.listings.show', $listing->id))
            ->assertValidRequest()
            ->assertValidResponse(200);
    }
}

The test running with a failure:

$> ./vendor/bin/phpunit --filter ApiListingControllerTest::testGetListingResponse
PHPUnit 8.5.14 by Sebastian Bergmann and contributors.

. E                                                                  2 / 2 (100%) 

Time: 3.34 seconds, Memory: 82.51 MB

There was 1 error:

1) Tests\Feature\Api\Listing\Spec\ApiListingControllerTest::testGetListingResponse
ErrorException: get-listings-id json response field data does not match the spec: [ required: {"missing":"address"} ]
Failed asserting that true is false.

./vendor/laravel/framework/src/Illuminate/Support/Traits/Macroable.php:114
./vendor/laravel/framework/src/Illuminate/Testing/TestResponse.php:1324
./vendor/hotmeteor/spectator/src/Assertions.php:83
./vendor/laravel/framework/src/Illuminate/Support/Traits/Macroable.php:114
./vendor/laravel/framework/src/Illuminate/Testing/TestResponse.php:1324
./tests/Feature/Api/Listing/Spec/ApiListingControllerTest.php:177

ERRORS!
Tests: 2, Assertions: 6, Errors: 1.

The test running successfully:

$> ./vendor/bin/phpunit --filter ApiListingControllerTest::testGetListingResponse                    
PHPUnit 8.5.14 by Sebastian Bergmann and contributors.

. .                                                                  2 / 2 (100%) 

Time: 4.22 seconds, Memory: 82.51 MB

OK (2 tests, 7 assertions)

These tests can be run manually, but the real power is running them automatically as part of CI/CD. That’s the next step.

Automatic Validation

After identifying the right tools, we were ready to use in an automatic fashion. Instead of manually using Spectral and Spectator, we included them in our existing CI/CD workflow. For Spectral, we created a custom GitHub workflow:

name: Lint Specs

on:
  pull_request:
    branches: [develop]

jobs:
  lint-specs:
    runs-on: ubuntu-latest
    steps:
      - name: Check out api repo
        uses: actions/checkout@v2

      - name: Setup node
        uses: actions/setup-node@v1
        with:
          node-version: '12'

      - name: Install node packages
        run: npm install

      - name: Install spectral
        run: npm install -g @stoplight/[email protected]

      - name: Lint spec
        run: spectral lint --ruleset spec/.spectral.json --resolver spec/skip-resolver.js 'spec/reference/*'

      - name: Lint spec models
        run: spectral lint --fail-severity warn --ruleset spec/.spectral.json 'spec/models/*'

And for Spectator, no change was required. All Spectator tests could be automatically included as part of our existing PHP unit tests (./vendor/bin/phpunit)!

Asynchronous Development

With all these pieces in-place, our backend engineers were able to confidently begin API development. In our Laravel-based world, that meant building API controllers and response/request objects while using the spec as a guide; but, even if we weren’t using Laravel, the process would have looked the same for any other stack. The key was having the right pieces in place to enforce the spec as the source of truth — this is also what gives this step the ability to be somewhat asynchronous. For us, the frontend (mobile) engineers defined the initial spec, the backend engineers implemented it, and occasionally collaboration was necessary to smooth out parts of the spec that needed to change.

Release!

Ah, the final step. Once our backend engineers finished their API implementation and it passed all testing, it was safely merged into master. But, we didn’t stop there. We wrote a few more GitHub workflows to create handy resources from the spec:

name: Generate Postman Collection

on:
  push:
    branches: [ master ]

jobs:
  generate-postman:
    env:
      REPOSITORY: {repository}

    runs-on: ubuntu-latest

    steps:
      - name: Check out api repo
        uses: actions/checkout@v2

      - name: Check out postman repo
        uses: actions/checkout@v2
        with:
          repository: ${{ env.REPOSITORY }}
          path: postman
          token: ${{ secrets.TOKEN }}

      - name: Setup node
        uses: actions/setup-node@v1
        with:
          node-version: '12'

      - name: Install openapi to postman tool
        run: npm install -g openapi-to-postmanv2

      - name: Install swagger cli
        run: docker pull jeanberu/swagger-cli

      - name: Replace spec $refs with definitions
        run: docker run --rm -v ${PWD}:/local jeanberu/swagger-cli swagger-cli bundle -r -o /local/spec/out/Aryeo-deref.v1.json /local/spec/reference/Aryeo.v1.json

      - name: Create postman collection
        run: openapi2postmanv2 -s spec/out/Aryeo-deref.v1.json -o postman/aryeo-postman.json -O requestParametersResolution=Example,exampleParametersResolution=Example,folderStrategy=Tags

      - name: Commit files
        continue-on-error: true
        run: |
          cd postman
          git config --local user.email "[email protected]"
          git config --local user.name "GitHub Action"
          git add .
          git commit -m "update postman collection"

      - name: Push changes to postman repo
        uses: ad-m/github-push-action@master
        with:
          directory: ./postman
          github_token: ${{ secrets.TOKEN }}
          repository: ${{ env.REPOSITORY }}

name: Generate SDKs

on:
  push:
    branches: [ master ]

jobs:
  build_swift_client:
    env:
      LANGUAGE: swift5
      REPOSITORY: {repository}
      PROJECT_NAME: AryeoModels

    runs-on: ubuntu-latest

    steps:
      - name: Check out api repo
        uses: actions/checkout@v2

      - name: Check out sdk repo
        uses: actions/checkout@v2
        with:
          repository: ${{ env.REPOSITORY }}
          path: sdk
          token: ${{ secrets.TOKEN }}

      - name: Install swagger cli
        run: docker pull jeanberu/swagger-cli

      - name: Replace spec $refs with definitions
        run: docker run --rm -v ${PWD}:/local jeanberu/swagger-cli swagger-cli bundle -r -o /local/spec/out/Aryeo-deref.v1.json /local/spec/reference/Aryeo.v1.json

      - name: Install openapi generator cli
        run: docker pull openapitools/openapi-generator-cli

      - name: Clear the sdk repo
        run: |
          sudo rm -rf sdk/*

      - name: Generate swift sdk
        run: docker run --rm -v "${PWD}:/local" openapitools/openapi-generator-cli generate -t /local/spec/codegen/templates/swift5 -i /local/spec/out/Aryeo-deref.v1.json -g swift5 --additional-properties projectName=AryeoModels,readonlyProperties=true --skip-validate-spec -o /local/sdk

      - name: Remove extraneous files
        run: |
          cd sdk
          sudo rm -rf .openapi-generator
          sudo rm -rf docs
          sudo rm -rf ${{ env.PROJECT_NAME }}/Classes/OpenAPIs/APIs
          sudo rm .openapi-generator-ignore
          sudo rm ${{ env.PROJECT_NAME }}.podspec
          sudo rm Cartfile
          sudo rm git_push.sh
          sudo rm project.yml
          sudo rm README.md
          sudo touch README.md
          sudo chmod 777 README.md
          sudo echo "# Aryeo SDK" > README.md

      - name: Commit files
        continue-on-error: true
        run: |
          cd sdk
          git config --local user.email "[email protected]"
          git config --local user.name "GitHub Action"
          git add .
          git commit -m "update sdk"

      - name: Push changes to sdk repo
        uses: ad-m/github-push-action@master
        with:
          directory: ./sdk
          github_token: ${{ secrets.TOKEN }}
          repository: ${{ env.REPOSITORY }}

These workflows produce our API’s documentation, an interactive Postman collection, and even an SDK for the Swift/mobile app. And, as the API is updated in the future, we’ll continue to get these byproducts for free! Not bad for a few lines of code.

Staying Spec-Driven

At first, adopting spec-driven development felt a little clunky. We had to set up numerous tools and adjust the way we worked as an engineering team. However, the proof is in the results. With spec-driven development, we’ve been able to iterate on our API while keeping backend and frontend engineers in sync. All of our API decisions begin with our specs and flow into our implementations. This has reduced code duplication, unified data formats, and made for a really pleasant developer experience. We feel like spec-driven development is the future, and we hope this post has given you a starting point if you’re considering a switch!