Blogs on Dillon Shook

Tradeoffs in Testing

Wed, 17 May 2023 07:37:42 +0000

I’m finally getting around to putting down in words the testing philosophy I’ve developed over the years. Some of it might rock the boat a bit, other parts may be obvious to some folks, but what I’d like you most to take away is a reflection on your own testing philosophy and the tradeoffs your making. There’s no free lunch.

Choosing how much time to invest in testing is all about balancing how confidently you can say your program is doing what you intend to and how fast you can modify or add to your program. Writing tests takes time that could otherwise be spent adding new features. When requirements change in a project with tons of tests, tests will break and that means spending time debugging and evaluating what should change in the tests as well. The reason to write tests though of course is to save time from manually verifying everything still works after changing the program.

So the testing approach that strikes the right balance of value for time spent is extremely dependent on the type of project you’re working on. It really irks me to read articles on testing that bake in lots of assumptions about the project, team size, product needs, etc. Their conclusions could be totally wrong for your situation! The obvious example is that you need way more tests for an MRI machine than for your weekend web app side project.

In general I think these are the four big areas that determine what your test coverage should look like.

Lets break down each one:

When the team size is small each developer knows a greater percentage of the codebase and is less likely to inadvertently break something because they didn’t know about it. As the team size increases, developers tend to focus on smaller parts of the codebase so having increased test coverage can catch bugs when cross cutting changes are made that affect parts of the system that the author has no awareness of.

Product maturity is all about how big your product is and how often the requirements are changing. In a mature product the requirements and tests will be changing less frequently, therefore having a longer time that they can catch issues “for free”, therefore better value for the time it took to make them. They also help everyone remember all the different features of the product that are buried away and all the people that built them have since left. As the product becomes so mature that it’s put into maintenance mode, you hit a point of diminishing returns where adding tests becomes pointless since nothing new is changing to break it. On the opposite side of the spectrum when you’re working on a new greenfield project, requirements are being figured out and are changing constantly, lots of code is getting thrown away, and the tests you write will soon become obsolete so it’s also not a great time investment.

When you’re writing a package, library, or something else that’s low in the stack you’re going to have a lot of Dependent code. When you use other peoples code yourself and it breaks you get sad. Don’t make people sad by not testing your code that other people depend on.

Cost of failure is the big one here that I think warrants an exponential growth in the test coverage. This is the heart of why the MRI machine in the previous example needs way more tests than the side project. The cost of failure in the side project is just your own time, in the MRI it’s potentially someones life and a multi-million dollar machine. The cost of failure is somewhat intertwined in the other metrics but there are large products with lots of people working on them that still have a way lower cost of failure than something like a mars rover.

So how do all these factors combine together? My mental model is to just average them all together. So for example when I built Macrocosm it was a solo project where requirements were changing constantly (low product maturity), there was no dependent code, and a low cost of failure. All these together tell me that very low test coverage is appropriate, and that’s what I had. Could I have caught bugs with more tests? Yes. Would it have been worth it for the amount of time it would have taken to write and maintain them? No.

Another arbitrary example might be if you’re on a small team, working on a legacy product, with little to no dependent code but that still brings in a ton of money for the business. You’ll have to pick the arbitrary numbers on the scale for yourself, but to me this warrants a moderately high amount of testing mainly driven by the cost of failure factor, but tempered by the small team and no dependencies.

With a rough idea of how much to test, lets talk about some goals for actually writing tests

Testing goals

The overarching goal of your test suite should be to build your testing suit of armor piece by piece where each test only overlaps with other tests to ensure there aren’t any gaps. You don’t want each of your tests asserting the same thing over and over (like testing authentication for each route in a web api). That just adds bulk and slows you down when requirements change.

Your knight starts out with no testing armor. They’re quick on their feet and can dance around other fully clad knights, but they’re vulnerable. When it comes time to add some protection think about what’s most vulnerable and focus on that first. What’s the most vulnerable part of your application? Really stop and think about what the most likely scenarios are for bugs and figure out how to write a test to catch them. If the scenarios are improbable think twice about if you really need a test. For example, I’ve come across a number of tests for simple React components where all they do is mount the component and assert it rendered. There wasn’t anything complicated the components were doing that was a likely scenario to break, so we decided to shed some weight.

For this reason I think code coverage percentage is generally a dumb metric to strive for in all but the highest testing need projects. What’s the value in testing all the code in your project that is so straightforward that the only way it could break is if it was intentionally changed for new requirements? Ideally you’re striving for the first part of the mantra “Either your code is so simple that there are obviously no bugs, or your code is so complex that there are no obvious bugs”. From what I’ve seen in projects with arbitrary test coverage goals, people spend more time gaming the coverage metric than the time saved coming up with useful tests that catch bugs.

Back to the armor, your tests should also come in layers. Under the plate armor there’s a padded gambeson which provides more protection and softens blows at a fraction of the weight. By architecting your code in layers where each layer has a single responsibility it makes testing so much easier. For example in a typical web api backend, if your code has different layers to handle data fetching/persistence, business logic, and presentation transformations, each layer can test its own responsibilities without worrying about the rest. Thinking about how to structure your code so it’s easier to test. This often means isolating complex business logic in a pure function that can be rigourously tested. Depending where you are on the test coverage scale, you might determine tests for the complex business logic layer are sufficient and everything else is straightforward enough.

You might ask though, how can each layer not worry about the rest if each layer depends on other layers? This is where Dependency Injection really shines. You can build out a suite of reusable fake implementations to inject into the layer you’re testing against. Building out full testing implementations of a layer may take longer initially but the time should be well spent when you can reuse them across all your tests, and have a single place to update when the layer behavior changes. Contrast that with the Jest mocking approach that monkey patches dependencies in each test and are nearly guaranteed to break across many tests when the layer behavior changes.

Another way to think about it is to think of your test suite as its own standalone application. It’s “just” another program, which happens to have the requirements of verifying what another program does. Don’t throw out all your best practices and architecture for writing a good app when it comes to the test suite! I’ve seen test suites where there’s tons of boilerplate setup code repeated everywhere and lots of inconsistencies between tests. The test suite has to be maintained just like the application.

Ideally you want your tests as decoupled from the implementation as possible so it is easier to maintain. You want to be able to assert on just the important logic (why your application exists in the first place) and have the tests remain the same as you refactor the implementation so you’re sure it behaves the same. This of course is more feasible the higher up the testing pyramid you go, but still important in unit tests as well. Test the interfaces, not the implementation details.

If the dev writing the code is also writing the tests, they have certain blind spots, assumptions, and probably a bit of laziness (*cough* guilty as charged) about how the feature will be used and what scenario to test. Automated testing can get you far, but it’s still not a complete substitute for real users prodding it. As an exercise, take the time to try and break your own code when you write it. Throw lots of data at it. Throw some naughty data at it. Fuzz it. Try it in a different environment (hardware/OS/browser/etc). Get annoyed at it when it takes too long. I can almost guarantee you’ll find something you didn’t think of in the routine testing motions.

The best time to find bugs is right after you wrote the code while everything is still fresh in your head. John Carmack talks here about running code with the debugger right after you write it to verify it’s working like you expect it does. I’m similarly amazed that more developers don’t do this since it’s not only such a good way to catch bugs as soon as possible, but also a great way for Upgrading your Brain Virtual Machine. This is harder to do in environments where there’s lots of async and framework code, but it’s still worth doing when writing complex functions and will give you a better understanding of what all is going on.

Wrapping up

Hopefully reading this has sparked some introspection on your own testing philosophy. I didn’t want to rehash specific testing methodologies that are well covered by other articles but instead get people thinking about the cost benefit analysis of the methods as applied to their situation. Don’t blindly follow advice from people. Especially me

Testing Next.JS with Cypress and Mock Service Worker

Tue, 08 Nov 2022 07:58:08 +0000

If you’re running a Next.js app and want to test it with Cypress you probably noticed a problem when it came time to test a server side rendered page. There’s no built in way to mock out requests made in getServerSideProps. When I ran into this and searched around for solutions I found this article that has a good explination of the problem (and diagrams), but I wasn’t satisfied with any of the solutions.

Here’s a quick overview of the problem, there’s no way to intercept requests made from the Next.js node process:

The article offers one solution of running the Next app inside the Cypress test runner and using Nock, but I didn’t like that solution since it seems brittle to run in CI and also you would have to have separate mocking libraries for requests made on the frontend. The article then goes on to describe a solution with Mock Service Worker which looks appealing because it can run on both the server and the client, but there’s still the catch of controlling it from the Cypress tests which the author doesn’t appear to have actually solved.

We were in this “trying to find a good way to mock” boat a few months ago and despite finding a few articles on setting up MSW with Cypress and Next there wasn’t anything that satisfied my goals. The the vercel MSW example has the basics of setting up MSW doesn’t include Cypress in the setup and doesn’t have any suggestions on how to control MSW from a test.

When setting up a mocks for our tests I had a few main goals:

One set of mocks for both server and client
Easy to use in tests, and easy to write handlers
Able to have tests set up specific mocking scenarios
Not rely on complicated CI environments
Use as few packages as possible
As simple as possible, but no simpler

To recap, the couple of problems that make this hard are:

Any network calls made in getServerSideProps are made in a node process, where the requests made from the browser are in a separate process
When testing, mocks must be set up before a page loads to guarantee all requests (including ones made while server side rendering) are mocked

The Solution

What I came up with certainly isn’t perfect (see Gotchas below) but it satisfies the goals, works within the constraints of the versions of Next (12.x), Cypress (10.x), and MSW (0.44) we have, and is certainly better than our previous setup.

The solution I came up with uses cookies and custom next api’s to trigger loading the mocks for tests.

With this setup your test code will be nice, simple and look like:

describe("a test", () => {
  it("does something", () => {
    cy.addMocks("mockFileName");
    cy.visit(`/a-path-to-test`);

    //then assert and interact as you would
  });
});

And the handler files that will be used for server and client requests just export arrays of MSW handlers:

import { RequestHandler, rest } from "msw";

export const handlers: RequestHandler[] = [
  rest.get("*/some-api-to-mock", (_req, res, ctx) => res(ctx.json([]))),
];

The Details

There’s a lot of different stuff happening here in lots of different places so lets go through it step by step.

The entrypoint for setting up MSW is in /pages/_app.page.tsx which creates either the server side or client side MSW instance. It checks to see if one is already created to prevent creating duplicates which is important to not confuse which mock handlers are on which instance. There are also a couple environment variable checks to a) make sure MSW doesn’t run in production and b) make sure it doesn’t run in normal development unless you want it to. More on that in the running it section. Finally, it tracks the loading state of the mocks and renders a loader to prevent pages making requests before MSW is initialized.

_app.page.tsx

const AppPage = ({ Component, pageProps, router }: AppProps) => {
  const [mswState, setMswState] = useState<"unused" | "loading" | "ready">(
    "unused"
  );
  // MSW requires initialization of its mock server inside the server code so it can intercept requests being
  // made by Next in getServerSideProps. Unfortunately it cannot be initialized from Cypress since it's
  // running in a separate process.  This should really run when the server starts up (not first request like it is)
  // But there's not an easy way to do that now https://github.com/vercel/next.js/discussions/15341
  if (process.env.NEXT_PUBLIC_IA_ENV !== "production") {
    if (process.env.NEXT_PUBLIC_MOCKS_ENABLED && mswState === "unused") {
      setMswState("loading");
      const mockImport = import("../src/mocks");
      mockImport
        .then(({ initMocks }) => initMocks())
        .then(() => setMswState("ready"));
    }
  }

  if (mswState === "loading") {
    // If MSW is enabled we must wait for it to start up before rendering to make sure all requests
    // are captured for consistency
    return <Loader />;
  }

  // normal return
};

The mocks index file is built off the example one but with the additional checks to make sure there’s only a single instance of both the server and client MSW instance, as well as loading test handlers if the cookie is set.

/src/mocks/index.ts

import Cookies from "js-cookie";

async function initMocks() {
  if (typeof window === "undefined") {
    //Make sure there's only one instance of the server
    if (typeof global.serverMsw === "undefined") {
      const { server } = await import("./server");

      global.serverMsw = {
        server,
      };
      console.log("Initialized SERVER msw");
    }
  } else {
    //Make sure there's only one instance of the worker
    if (window.msw === undefined || window.msw.worker === undefined) {
      const { worker } = await import("./browser");

      window.msw = { worker };

      console.log("Initialized BROWSER msw");

      const mockFile = Cookies.get("cypress:mock-file");
      if (mockFile) {
        const nextRequestRegex = /\/_next.*/;
        await worker.start({
          onUnhandledRequest: (req) => {
            // Don't warn about _next api call's not being mocked
            if (req.url.pathname.match(nextRequestRegex)) return;
            console.warn(
              "[MSW] unmocked request:",
              `${req.method} ${req.url.pathname}`
            );
          },
        });

        const allMocks = await import("mocks/handlers");
        const handlers = (allMocks as any)[mockFile].handlers;

        worker.use(...handlers);

        console.log("Added browser handlers for " + mockFile);
      } else {
        worker.resetHandlers();
        worker.stop();
      }
    }
  }
}

export { initMocks };

Both server and client instances initialize using the global handlers file which handles all the general requests the app makes that aren’t that page specific, and can be overwritten by test specific handlers.

src/mocks/browser.ts

import { handlers } from "./globalHandlers";
import { setupWorker } from "msw";

export const worker = setupWorker(...handlers);

src/mocks/server.ts

import { handlers } from "./globalHandlers";
import { setupServer } from "msw/node";

export const server = setupServer(...handlers);

src/mocks/globalHandlers.ts

import { RequestHandler, rest } from "msw";

export const handlers: RequestHandler[] = [
  rest.get("*/some/api", (_req, res, ctx) => res(ctx.json({}))),
  ...otherHandlersYouHave,
];

We now need a way to tell our Next.js backend that we want to start mocking for a specific test, and then be able to reset the mocks after the test is done. By setting up a Next API route we can do just that!

The first api lets us add a mock file and start MSW on the server. A lot of it is error handling but the main part is importing the handlers index file, starting MSW, then the server.use(...handlers) call to add the mock handlers. The API also adds a Set-Cookie header which is important for using this setup outside of Cypress, but more on that later.

pages/api/test-mock/add.api.ts

import { NextApiRequest, NextApiResponse } from "next";

import { RequestHandler } from "msw";
import { serialize } from "cookie";

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse
) {
  if (process.env.NEXT_PUBLIC_IA_ENV === "production") {
    res.status(405);
    return;
  }

  try {
    res.setHeader("Cache-Control", "no-cache");

    const fileName = req.query.file;
    if (!fileName || typeof fileName !== "string") {
      res.status(400).json({ status: "Missing file param", fileName });
      return;
    }

    const allMocks = await import("mocks/handlers");
    const handlers: RequestHandler[] =
      allMocks[fileName as keyof typeof allMocks].handlers;

    if (
      typeof global.serverMsw === "undefined" &&
      process.env.NEXT_PUBLIC_MOCKS_ENABLED
    ) {
      //Auto start up MSW if it hasn't been yet.  This can happen if this is the first request
      //from cypress before a page has loaded
      const { initMocks } = await import("mocks");
      await initMocks();
    }

    if (typeof global.serverMsw !== "undefined") {
      global.serverMsw.server.listen({ onUnhandledRequest: "warn" });
      global.serverMsw.server.use(...handlers);
      res.setHeader(
        "Set-Cookie",
        serialize("cypress:mock-file", fileName, {
          path: "/",
          sameSite: "lax",
        })
      );

      res.status(200).json({
        status: "Added Server Handlers",
        handlers: handlers.map((h) => h.info.header),
      });
    } else {
      res.status(400).json({ status: "Mock server not initialized." });
    }
  } catch (e: any) {
    if (req.query.file) {
      console.error("Error trying to add mock " + req.query.file);
    }
    console.error(e);
    res.status(500).json({ error: e.toString() });
  }
}

The reset api is very straight forward to stop the server MSW instance

pages/api/test-mock/reset.api.ts

import { NextApiRequest, NextApiResponse } from "next";

import { serialize } from "cookie";

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse
) {
  if (process.env.NEXT_PUBLIC_IA_ENV === "production") {
    res.status(405);
    return;
  }

  try {
    res.setHeader("Cache-Control", "no-cache");

    if (typeof global.serverMsw !== "undefined") {
      global.serverMsw.server.resetHandlers();
      global.serverMsw.server.close();
      res.setHeader(
        "Set-Cookie",
        serialize("cypress:mock-file", "", {
          path: "/",
          sameSite: "lax",
        })
      );

      res.status(200).json({ status: "Reset Handlers" });
    } else {
      res.status(400).json({ status: "Mock server not initialized" });
    }
  } catch (e: any) {
    // eslint-disable-next-line no-console
    console.error(e);
    res.status(500).json({ error: e.toString() });
  }
}

Ok we now have most of the plumbing going but now we need to set up a way to use test specific handlers. I originally had dynamic imports set up so no index file would be necessary but there’s a limitation with webpack on how dynamic the path can be which means the whole path can’t be dynamic, only the filename. This meant every test handler file had to go in one directory and quickly became an unorganized mess. So to fix that (as well as fixing file change auto restarts which were broken with dynamic imports) we switched to a handler index file that exports all the test specific handlers. An important part here is that the name that they’re exported as is the canonical name you’ll use to add them in tests.

src/mocks/handlers/index.ts

export * as mockFileName from "./testName/mockFileName";

// ... there will be a whole bunch more for different tests and scenarios

The test specific handlers just export an array of MSW handlers which can be as complicated or as simple as your test needs. Remember that these handlers complement everything in the globalHandlers.ts file. New api’s can be handled here and global handlers can be overwritten for a specific test scenario.

src/mocks/handlers/testName/mockFileName

import { RequestHandler, rest } from "msw";

export const handlers: RequestHandler[] = [
  rest.get("*/exhibits", (_req, res, ctx) => res(ctx.json([]))),
];

Finally we need to integrate with Cypress to make it easy to add mocks so we’ll use custom Cypress commands to add and reset mocks.

cypress/support/commands.js

Cypress.Commands.add("addMocks", (fileName) => {
  //Set cookie is still necessary here even though the api sets cookies to get it
  //set on the right cypress browser instance
  cy.setCookie("cypress:mock-file", fileName);

  cy.request({
    method: "GET",
    url: `/api/test-mock/add?file=${encodeURIComponent(fileName)}`,
    failOnStatusCode: true,
  });
});

Cypress.Commands.add("resetMocks", () => {
  cy.clearCookie("cypress:mock-file");

  // Reset snooped requests and listeners
  cy.window().then((window) => {
    const worker = window?.msw?.worker;
    if (worker) {
      worker.events.removeAllListeners();
    }
  });
  mockedRequests = [];

  return cy.request({
    method: "GET",
    url: `/api/test-mock/reset`,
    // Turn off failing on status code for smoke tests that won't have mocks set up
    failOnStatusCode: false,
  });
});

let mockedRequests = [];

Cypress.Commands.add("startSnoopingBrowserMockedRequest", () => {
  cy.window().then((window) => {
    const worker = window?.msw?.worker;

    if (!worker) {
      reject(new Error(`MSW Browser worker not instantiated`));
    }

    worker.events.on("request:match", (req) => {
      mockedRequests.push(req);
    });
  });
});

/**
 * URL is a pattern matching URL that uses the same behavior as handlers URL matching
 * e.g. '* /events/groups/:groupId' without the space
 */
Cypress.Commands.add("findBrowserMockedRequests", ({ method, url }) => {
  return new Cypress.Promise((resolve, reject) => {
    if (
      !method ||
      !url ||
      typeof method !== "string" ||
      typeof url !== "string"
    ) {
      return reject(`Invalid parameters passed. Method: ${method} Url: ${url}`);
    }
    resolve(
      mockedRequests.filter((req) => {
        const matchesMethod =
          req.method && req.method.toLowerCase() === method.toLowerCase();
        const matchesUrl = matchRequestUrl(req.url, url).matches;
        return matchesMethod && matchesUrl;
      })
    );
  });
});

And in the e2e file we’ll make sure to reset the mocks after each test:

cypress/support/e2e.js

// Always reset the MSW mock overrides between each test so they don't pollute each other
afterEach(() => {
  cy.resetMocks();
});

Mocked Request Assertion

You probably noticed there’s a bit of extra stuff in the Cypress commands for snooping on network requests. The the official MSW doc says:

When testing, it may be tempting to write assertions against a dispatched request. Adding such assertions, however, is implementation details testing and is highly discouraged. Asserting requests in such way is testing how your application is written, instead of what it does.

I disagree with this take however since a big part what an application does is interact with APIs. You probably want to know if you’re screwing up a critical API call when testing. So it can be very valuable to assert that requests were made correctly, especially POST/PATCH requests.

To do this, add the following to your test body: cy.startSnoopingBrowserMockedRequest();

This will start capturing all the subsequent requests for the test that are mocked out by MSW. Important to note in the name, this will only capture the browser client side requests due to the MSW setup.

Once the test performs whatever action it needs to, you can then get all matching requests and assert like so:

cy.findBrowserMockedRequests({ method: "PATCH", url: "*/an/api/:params" }).then(
  (patchRequests) => {
    assert.equal(patchRequests.length, 1);
    const body = patchRequests[0].body;

    assert.equal(body.data, { something });
  }
);

The resetMocks command run after each test will reset the snooping and request log so tests don’t pollute each other.

Test Code

Once all that’s set up your test code will look like

cypress/e2e/specs/sample.js

import { dataOrIdsMocked } from "mocks/handlers/testName/mockFileName";

describe("a test", () => {
  it("does something", () => {
    cy.addMocks("mockFileName");
    cy.visit(`/a-path-to-test`);

    //then assert and interact as you would, using imported dataOrIdsMocked as needed
  });
});

Running It

You can set up your dev scripts however you want of course as long as the right environment variables are set up, but here’s a suggested way to set it up so locally you can just run yarn cypress which will start both the Next.js server and Cypress with all the config handled using the concurrently npm package.

package.json

{
  "scripts": {
    "dev:mocked": "NEXT_PUBLIC_MOCKS_ENABLED=1 yarn dev",
    "dev": "concurrently -n NEXT,TS -c magenta,cyan \"yarn dev:start\" \"yarn ts --watch --pretty\"",
    "cypress:config-and-open": "node_modules/.bin/cypress open --config specPattern=cypress/e2e/specs",
    "cypress": "concurrently -n Dev,Cypress -c blue,yellow  \"yarn dev:mocked\" \"yarn cypress:config-and-open\""
  }
}

A real nice bonus about getting MSW set up like this is using it outside of testing just for developing new features where an API isn’t finished yet. Devs can just run yarn dev:mocked which will start up Next with the mocks enabled, they can set up a handler file the same as they would for a test scenario, manually go to /api/test-mock/add?file=mockFileName, and then use the app as normal with a new API mocked out. Once they’re done developing, that mock handler file can then be directly turned into a Cypress test.

Gotchas

As said in the intro, this setup is not without flaws that I wish had solutions. If you can think of solutions please leave them in the comments!

1. Mocks must be added before navigating to a page

Because mocks must be loaded before a page starts loaded so no requests are missed, they must be added before navigating to a page. The browser side setup loads the mocks as the page is loading.

So you can’t call cy.addMocks multiple times in the same test file unless you are visiting multiple pages in the same test.

cy.addMocks("some-mock-file");

cy.visit("page1"); // some-mock-file mocks added here

cy.addMocks("more-mocks");

// more-mocks haven't been loaded in yet
cy.get("[data-cy=abc123]").should("exist");

// now the more-mocks get loaded in
cy.visit("page2");

2. It’s hard to dynamically change mocks during or between tests

If you need the same endpoint to return different values throughout a test file there’s two strategies:

Create multiple mock files and add them in where needed.

cypress/e2e/specs/sample.js

describe("a test", () => {
  it("does something", () => {
    cy.addMocks("mockFileName");
    cy.visit(`/a-path-to-test`);

    // ...
  });

  it("does something else", () => {
    cy.addMocks("mockFileNameWithDifferences");
    cy.visit(`/a-path-to-test`);

    //...
  });
});

This is usually the best course of action since it makes it clear what the mock setup for each test is. It does lead to lots of handler files with slight differences between them so you’ll need to use normal software best practices to not duplicate a whole bunch of stuff in each handler file. Util functions shared between handlers that return arrays of handlers are a good way to do that.

An alternative approach that works better for some kinds of tests is to use one handler file, but maintain state in it as requests are made. See next point for the caveat about this but here’s an example of what that looks like:

/mocks/handlers/statefulHandler.ts

import { RequestHandler, rest } from "msw";

let testState = 0;

const returnObjects = [
  { status: 404, body: { error: "Testing error" } },
  { status: 200, body: { someObject } },
  { status: 200, body: { someOtherObject } },
];

export const handlers: RequestHandler[] = [
  rest.get("*/jobs", (_req, res, ctx) => {
    const ret = returnObjects[testState];

    if (testState === 1) {
      testState = 2;
    }

    return res(ctx.status(ret.status), ctx.json(ret.body));
  }),
  rest.post("*/jobs", (_req, res, ctx) => {
    testState = 1;
    return res(ctx.json({ whatever }));
  }),
];

3. State is tricky in handlers

In the above example there’s a big gotcha: the handler state is independent between the client and server. So if some requests are made in server side rendering then the browser makes some requests, each will start at the initial state. This is of course because of the two different instances of MSW, one on the server, one on the browser.

4. Mock files get imported multiple times

The mock handler files get imported by several places like when the page loads, and when the server side mocks are enabled via API. If the tests are importing ids or other data from the handlers, these must be static values. One thing that tripped us up was using sequenced id’s in factories created by the fishery library. If you try this you’ll find that the test import will get a different value returned than the mocks are using.

The End

Hope you found this useful, at the very least as another approach to consider. I spent way more time than I thought getting Next, Cypress, and Mock Service Work set up in a way I was ok with. There were some good articles out there on basics but nothing that covered everything I wanted so here I am filling in some gaps.

If you think you have a better way, or something to improve upon with this approach please let me know!

What it Takes to Make a Game by Yourself

Thu, 15 Sep 2022 16:37:00 +0000

The short answer is perseverance, skills and the ability to teach yourself new ones, creative problem solving, learning from mistakes, and a looot of time. That’s my take anyway. Read on if you’re curious where I’m coming from!

The Rest of the Intro

I recently released Macrocosm which is a mobile game that takes you from atom to galactic empire across seven interconnected stages where making progress in one stage gives you a boost in the next. This post is a deep dive into the (nearly) four years of free time I spent making it. Part catharsis, part brain dump while things are still mostly fresh, but mainly giving back to the game dev community, this post is for anyone currently making an indie game or thinking about it.

First, let’s start with a few stats, because who doesn’t love those, and it also should give you a better sense of the scope of the project.

By the numbers

First commit June 2nd 2018.
594 Trello cards created to keep track of tasks and bugs
677 days worked on
2097 commits (so far)
Most productive days were 2/23/2020 and 9/20/2020 with 13 commits each.

Lines of Code

Without 3rd party libraries & dependencies:

--------------------------------------------------------------------
Language          files          blank        comment           code
--------------------------------------------------------------------
C#                  321           8377           2789          39958
HLSL                  9            102              5            517
Javascript            1              7              1             44
Bourne Shell          1              8              5             23
JSON                  1              0              0              9
--------------------------------------------------------------------
SUM:                333           8494           2800          40551
--------------------------------------------------------------------

With 3rd party code:

--------------------------------------------------------------------
Language          files          blank        comment           code
--------------------------------------------------------------------
C#                 1031          23359          23148         117836
HLSL                201           1077            280          18038
JSON                 26              1              0          10365
Swift                 1             65             10            375
Objective C           1             24              7            109
Objective C++         1             18              5             62
Javascript            1              7              1             44
XML                   1              2              0             29
Bourne Shell          1              8              5             23
C/C++ Header          1              1              0              4
--------------------------------------------------------------------
SUM:               1265          24562          23456         146885
--------------------------------------------------------------------

Back of the Napkin Hours Worked

Using the git-hours tool (and having to fix a bug in the repo along the way!) I used an estimate of 30 minutes before first commit in a session to come up with a very back of the napkin estimate of 1,366 hours. This only takes into account the coding time and the art I did myself, not any of the off computer design, testing, organization, etc. If I had to guess I’d tack on another 300 hours or so for all that.

Budget

I tried to be pretty frugal with how much I spent. Both because it’s my hobby side project, and because part of success to me was just being able to break even on costs which is easier when you have less of a hole to dig yourself out of :)

Expense	Amount
3rd Party Assets & Plugins	$264.60
Business Costs	$1,089.00
Contractors	$3,781.88
Web, Email, & LFS Hosting	$162.98
Apple Fees	$340.57
Unity Cloud Build	$210.32
——————-	———
Total	$5,849.35

Early Project Times

After coming to the tough realization I wouldn’t be able to finish my previous project Solaria Tactics that I’d been working on for a couple years I took some time off working on side projects. Around that time Universal Paperclips spread through the office and internet like fire and I loved how it kept you on your toes for what would be next and what new gameplay would be unlocked. I still had the drive inside of me wanting to release a game that I could be proud of and actually release to the world, so it didn’t take much of a spark to get my brain thinking of game ideas again. Like Universal Paperclips, I wanted to take the player from something humble and small to something gigantic and have a variety of gameplay along the way to keep it all fresh and new.

So I wrote down a few design pillars:

Variety of gameplay
Connected stages
Can focus on your favorite stage, don’t let the player get stuck
Lower stage progression acts as a force multiplier for later stages

then brainstormed some ideas on what the stages would be, then got back to work! :D Two days after the first commit I had this

Your browser does not support the video tag.

Which if you can squint hard enough, still forms the basis for stage one today! You can also see my fascination with MASSIVE NUMBERS that was coming from paperclip land. More on that later…

At this point in the process, I was thinking I’d spend about 1-2 months on each of the 7 stages and wrap up the project in about a year. Spoiler alert: that didn’t happen lol.

One of the early challenges was figuring out how the stages would work together. At one point I was thinking there would be some linear path where the game would automatically bounce you between stages as you made progress in some sort of Tower of Hanoi-like pattern. This got tricky to design very quickly as I was writing it down, and luckily was convinced by a friend to change it pretty early on and let the player decide when to switch stages.

There was some interview with Chris Taylor about the making of Supreme Commander (still one of my favorite games) I read a long time ago where he was describing that as the game was being made, naturally everyone got better. The art made 10 months ago wasn’t as good as the the art being made now. That lead them to working on the game in passes where they would try to make things as good and fleshed out as they can, then go back in another pass and improve everything.

So this is how I thought about working on Macrocosm. I would first make all the seven stages in a first pass rough draft state where they had the basic gameplay I thought would work and very placeholder everything else, then do another pass to refine, and repeat till release! I did follow that model throughout the project and think it worked pretty well for the project overall, especially given the connected nature of it all. I think it did save me from throwing away work and art that otherwise would have gone to waste if I would have tried to polish or finalize things too early.

Stage 2

Less than a month after starting the project I was onto drafting the second stage

Your browser does not support the video tag.

Working on the snake movement code was one of those things where it’s easy to get the basic version of it working, but surprisingly nuanced and complicated to get it to feel and work exactly like you want it to. I ended up coming back to it a few times over the course of the development to refine. In this first version you see it’s doing the approach of having each part of the chain lerp towards it’s immediate neighbor each frame. The problem with this approach is that as the snake does a bunch of tight turns the pieces of the chain down the line start to smooth out and follow a straighter line path instead of exactly where the head actually went. The other problem is that it’s hard to make the head piece follow a nice curve while turning so you can’t immediately turn back on yourself and break the chain which feels bad. I ended up with a sparsely populated circular buffer of points where the head went and then code to figure out which point to go to next. Lemme know if you know a better way :)

Stage 3

Your browser does not support the video tag.

The first basic version of stage 3 came along pretty smoothly, though this was my first foray into the world of getting audio syncing up with gameplay. This is another one of those “oh that sounds easy” problems that is very much not. I found these guides that were immensely helpful though. As the rhythm game crash course says:

In rhythm game first you have to get the backing track to line up with the first “note” (what is that depending on your game) and the rest will stay correct UNLESS the game lags or the audio lags. 90% of the time the game lags and audio went ahead of the game since audio is not in the same processing unit with the game anymore after the play command. The lag requires separated resolution and I will not talk about it right now.

Audio in Unity is “fire and forget”. When you ask Unity AudioSource to play it will take variable amount of time and play whenever it feels like.

The really tricky part is that your gameplay code only runs ever X milliseconds based on your frame rate, but the music is continuously playing. So between each time the update loop runs there’s a different amount of music that has played. It takes some careful math calculations and using

  public double MusicTimeElapsed {
    get{ return (double)musicSource.timeSamples / musicSource.clip.frequency; }
  }

to synchronize gameplay code to spawn things on the beat of the song.

Stage 4

When I was first started thinking about what I’d do for stage four I went down this big rabbit hole thinking of a platformer where you controlled a creature going around collecting DNA points sonic style (or perhaps EVO style) and you could choose which traits to unlock to make your creature hold its breath longer, jump further, or go faster and let you get to other areas of the map like a metroidvania. I went so far as to sketch out a whole map of how it would look before coming to terms with how hard that all would be to implement and play well. Thankfully I made that realization before starting to build any of it and brainstormed other ideas.

I ended up going with a roguelike partially inspired by the Evolution board game that fit my goals for the stage:

Have unbounded replayability
Consumes lives naturally
Have some creature customization
Balance skill with upgrade power

Your browser does not support the video tag.

As usual, I wanted to go crazy with lots of creature adaptations that you could choose from that have gameplay impact, but more on that later. Along the way doing research for the stage I did find this awesome zoomable tree of life that you can find any species in and see how they relate to other species.

Stage 5

I played way too much Desktop Tower Defense in middle school computer class which kickstarted my love for the tower defense genre so I knew I wanted to do my own spin on it for Stage 5 from the beginning.

It didn’t take too long to come up with the idea of layering on a resource collection aspect to the tower defense mechanism, and then using your population (lives traditionally in TD) as a necessary resource for upgrading your towers on top of that.

Your browser does not support the video tag.

The only notable challenge getting this stage set up initially was getting the pathfinding set up for multiple starting locations with different destinations, and then handling all the cases where you need to update (and block creating new towers when you’re putting it on top of an enemy). The Red Blob Games articles on pathfinding were helpful, particularly the one dedicated to tower defense.

Stage 6

Whew, onto stage 6 huh. How many more stages do I have to do again? Only one more? Ok, we can get through this, even though it’s already February 2019 and past when I originally thought I’d be done with the whole game. Oh well, time to make a mini civ game!

I’m one of the millions of Civilization fans out there so it was a no brainer for me to base this stage off of it. However I did know that creating a combat system with lots of units to design, and an enemy AI to fight against was going to be way too much for me to do for just this stage. So I decided to focus on my favorite parts of civ like the exploration, city placement & management, and tech upgrades. This nicely danced around the tech intensive problems but also the design problem of “what happens if you lose” in a combat oriented game since I didn’t want the player to ever have to restart.

The Catlike Coding hex map tutorials gave me a good boost on getting the initial map set up and world generation going but it still took a good bit of time to customize it and convert it from 3d to 2d.

It predictably took even more time to then write all the actual gameplay stuff on top of that like the first pass at city management, buildings, technology system, scouting, and the first pass at rivers and roads. After 4 months I had this:

Your browser does not support the video tag.

Funnily enough, I got stuck on the triangle controls for city management and asked this question on the Mathematics stack exchange site and got a predictably dense math explanation that I couldn’t follow and had to work it out myself.

Stage 7

Alright, onto the grand finale of the last stage. Something fun in a galaxy setting should be easy right? Here’s what I was thinking of as the constraints for the stage:

Primary Constraints:

Able to build it in reasonable amount of time
Tie in stage 6 technology research
End as a nice capstone for the game
Primary goal of exploring the galaxy

Secondary Constraints:

Use mass & energy as resources
Have gameplay focusing on automating tasks that allow you to spread throughout the galaxy in an exponential fashion
Work between solar system & galaxy scales
Potentially have multiple endings/strategies for replayability
Have nice space visuals so it’s not all just UI manipulation

Having come off a long Factorio play through not long before getting to this stage I naturally was caught by the bug to make an automation style game at this point. I was imagining building up your colonies to mine resources, build probes and colony ships that would then colonize other nearby systems and repeat.

I wasn’t quite sure how this would all fit together at first though so I started by building the part I was sure I wanted which was procedural galaxy generation and the 3 main zoom levels going between planet, system, and galaxy.

Your browser does not support the video tag.

Since the game is science based I wanted the galaxy generation to be grounded in reality even if there were liberties to be taken for the sake of gameplay. I did a fair bit of research on all the nerdy galaxy formation stuff like: stellar classification, types of planets, gas giant classification, solar main sequence, distribution of luminosity classes, some old world building site, and a bunch of others. I settled on using GURPS Space which is a tabletop role playing guide as a starting point. It has a lot of details on galaxy, system, and planet formation with a bunch of probability tables that I could translate into a galaxy generation algorithm and then adapt as needed to fit the gameplay.

Once all that was in a decent state I had to come back to actually making a game out of it which turned out to be harder than I thought. Prototype after prototype just kept feeling tedious, too complex, or just plain boring. This turned out to be the trickiest game design problem of the whole game. Just balancing all the constraints took so much trial and error to come up with a design I liked.

I ended up taking inspiration from Mini Metro with the added layers of mining a small number of resources, combining them into other resources which are used to build and expand your galactic empire. By focusing on a small number of resources each with a theme and then having the combinations follow what you might expect felt like it was pretty approachable and gave a good design space to work with. For example Iron’s theme is the base building resource, Xenon’s is engines and transportation so naturally the combination of Iron and Xenon is the Thrusters resource used by colony and transport ships.

Your browser does not support the video tag.

Once I got Stage 7 in a reasonable state it was time to go back to the beginning and start making content, work on the final art, and just refine everything. And oh yeah, by the way, it’s already a year and half in and the game is nowhere near being done :)

A Brief Interlude on Tools

Roughly at this point in the project (really mixed in throughout working on other stuff) I spent a good chunk of timing honing the tools of the project. As a solo dev you don’t want to spend too much time just working on tools but being careful with creating custom tools for your project and using the right 3rd party assets really is worth it’s time in weeks. I would also advise to be pretty picky about what 3rd party assets you choose to use because integrating them into your project, really learning how they work deeply, and even finding some bugs with them can really sap your time.

One last tidbit of advice is to try to research and decide on core assets and libraries as early as you can to save yourself the hassle of switching to different ones mid or late project. Graphics, object pooling, and tweening packages are all good examples of this.

Here are the notable ones I used with some notes.

I created my own Save Editor that was worth its weight in gold to be able to manipulate my save file outside of code to test all sorts of different scenarios and flip between different save files that were in different states and see what was going on.
Around this time I switched from SVG Importer to the Unity vector graphics package to render all the games graphics. SVG Importer wasn’t being supported anymore, had some memory leaks, and required different components depending on what you were doing. It took a bit to get everything switched over but I’m happy I did it. The vector graphics package is only around 3/4ths baked and then put on hold as most things are from Unity these days but I was able to get around its limitations. Having 9-sliced svg images would be real nice though.
I used StrangeIOC from the beginning of the project since I had used it on my previous project. It’s nice dependency injection that really helps architect your project in a clean way. The caveats are that I made a couple small modifications to it, there are a couple quirks with how to use it, and don’t use the mediator concept (just bind everything in views directly) but being able to inject data models, events, and services into components is reaaally nice.
Nice Vibrations is a great and easy to use package that does exactly what it says on the tin.
The Color Palette asset is decent and saved me some trouble but it’s nothing too fancy that someone who knows how to create custom editor windows couldn’t do without too much trouble.
I got Unity Flexbox to try to get something that could get me off of the Horizontal and Vertical layout groups which never do what you want and I despise. However, the flexbox asset had some issues of its own I can’t remember and I had to modify it anyways and didn’t use it everywhere. 5/10
I started off by integrating the Unity User Reporting package that integrates with their cloud service but their site kept being super slow for me, sometimes would just be down when trying to send it a new report (or very slow as well), showed the screenshot in the wrong aspect ratio, and their retention period is a woefully short 7 days. All of this greatly frustrated the web developer in me and so I just wrote my own. It took me like 3 days to get everything I used from Unity working in my own except it was super fast and had infinite retention. Anyways, I highly recommend having a system in your game to both capture exceptions and report them to a server as well as an in game way for players to send feedback. This was so valuable to have both for debugging issues (and being able to download the save file that caused them) as well as improve the game through player feedback. If you’re interested in any of this, send me a message. I should probably turn that into its own post and open source it.
Object pool simple and effective. Tweaked a very small amount.
Switched from iTween to LeanTween for performance.
Shapes is a great asset for any sort of dynamic vector shape rendering. I use it a bunch for lines, progress bars, circles, donuts, and arcs that are all nicely anti-aliased.

Mid Project

So at this point in the project all 7 stages have most of their basic systems and gameplay worked out, the tools and infrastructure of the game are pretty close, but it’s all developer art still and everything needs this tricky thing called content. I assume everyone who’s made a single-player game before knows how time consuming this actually is, but for a new solo dev I found out it’s really easy to grossly underestimate.

The trap I fell into was thinking that once the systems were done content was a matter of adding rules, data, and a few more scripts. In reality though it’s doing those things plus then the realization you need a little more variety so the gameplay systems need extending, then play testing, then not being satisfied and tweaking some more, then working through problems with integrating the art, then balancing all the rules and systems, all while fixing a bunch of bugs.

Speaking of art though, it’s an amazing feeling to see the first final (or near final) art in the game. The project just seems so much more real after adding it and it’s a great morale boost. I highly recommend finding an artist to hire if you’re not sure you can pull it off yourself. The look of your game really helps differentiate you amongst the thousands of other games out there to choose from. Being able to create the first versions of the art to experiment and prototype with is really important as well though. It’ll save you time and money trying different ideas with placeholder cheap assets before committing to creating the final versions.

One thing you might also notice in that before and after is that the big number score got removed. I got so far as to implement it in each of the stages with a nice animation ticker effect whenever you got more points, but that ticker effect combined with getting points all the time meant that the score was allocating memory like crazy constructing new strings constantly. I was starting to look into ways around that when I thought about it more and wasn’t really happy with the arbitrary nature of what gave you more score and how much. So I did the simplest thing and just removed it all which I’m happy with. An important skill, especially as a solo dev, is the ability to evaluate your ideas and get rid of the ones that aren’t working. Every single idea you have isn’t a good one, trust me on that :)

Another example of an idea that needed editing was making the content for the third stage which is the cellular scale with a rhythm tapping and sliding to music theme. I started out by thinking I could randomly or procedurally create all the cells to tap on while each song is playing to save me from having to manually create a beatmap ahead of time. The more I playtested that though the more I realized that procedural wasn’t going to work for the amount of time it would take to make it good. There needed to be predicable repeating patterns of cells to tap just like how in a song there’s repeating patterns that make up the structure of the song (intro, verse, chorus, etc). If the player fails a song by missing too many beats it should also be the same patterns so you can actually practice getting better. So after the realization that doing that procedurally wasn’t easy I just made a simple system that allowed me to create the beatmaps, and then it was just a matter of spending the time to create and test all of them.

There were plenty of other challenges and things to learn along the way while refining each stage and adding content. On the creature stage I had to figure out how to animate the dino’s and promptly ran into the limitation of the vector graphics package that doesn’t really work with any of Unity’s other sprite animation systems. After doing some research on options I landed on using Anima2D for its bones and IK support even though it doesn’t seem to be supported or maintained anymore.

Your browser does not support the video tag.

Adding support for the notches at the top of phone screens was another unexpected challenge. Luckily there’s a nice Notch Solution out there that does most of the hard work. Making sure that all the game UI worked with different aspect ratios and while compressed in with the notches took a lot of bug fixing though.

A good forcing function for keeping the game simple while designing it was adding tutorials for each stage. It became pretty obvious that if I couldn’t explain what to do in a few pop up tutorials then I had made the game too complicated. This mainly was an issue on the last two stages building a civilization and a space empire which required a bit of simplification so I could try to explain to the player how they worked. In general though the tutorial system was yet another system that turned out to be more complicated than I originally thought it would need to be. It needs to handle all sorts of events from the game to trigger tutorials, pausing and unpausing the game correctly (don’t want to unpause the game if it’s already paused while a tutorial is shown), handling multiple tutorials trying to open at once, as well as saving and restoring the tutorial state when the game quits and then reopens.

The last pain in the butt around this time in development was getting automatic builds created every day for both iOS and Android. This was really helpful to be able to test on actual devices and catch bugs that were device specific as soon as possible (usually ones with touch input handling). I did end up paying for Unity Cloud Build since it’s obviously made for this exact thing. The build process for mobile apps is way more complicated to set up on your own computer than it should be so you’re kinda forced into using a cloud solution. But even with the hosted solution it’s not a walk in the park. You have to create custom scripts around accessing the build version inside the game and uploading the builds to the app store. Just setting up the certificates for iOS is an absurd back and forth process.

Finish it

At this point in the project I was feeling like you’re probably feeling reading this: “This took way longer I thought and I’m ready to be done with it” :)

But as with any passion project it’s hard to release something when you know it could be better and you see all the little flaws. So on goes the polishing, optimizing, tweaking, balancing, play testing, iterating, and bug fixing. I’m curious to know if anyone’s tried to quantify development time as a function of game length is but I’d be willing to bet it’s more than linear. Full playthroughs take longer and there’s just more of everything to test and tweak like you can see here in the before and after of the city management UI.

More code also means more places that can be slow which is important to pay attention to on mobile devices. This is another section that could be a whole post of its own but here’s my couple Unity optimization tips:

Get really familiar with the built in profiler, and add lots of custom samples with UnityEngine.Profiling.Profiler.BeginSample("My Sample");
Hunt down all the things that are allocating memory every frame. The usual suspects are creating strings, starting coroutines, and dictionaries keyed by enums need equality comparers.
Know your data structures, use HashSet’s instead of array’s if you’re mainly doing contains checks.

Getting the word about about your game is tough like you might imagine. I’m only barely qualified to talk about it myself so instead of me writing lots more words I’ll point you in the direction of Chris Zukowski’s how to market a game blog that has lots of useful info. Marketing for mobile games does seem to be a different beast though, some of which seems to involve questionable ethics that I don’t want any part of. What you should be part of instead is your local game developer community! It’s super helpful to find your peers and participate helping others with their struggles and in turn getting help with your own struggles. There were several talks given by the local group here that opened my eyes to things I hadn’t even thought about before.

If you are into marketing and interested in working on commission though definitely feel free to contact me through the email in the sidebar.

So at some point you just need to pick a date to release the damn thing. I gave myself the deadline of releasing the week of Pax East so I could do some guerilla marketing while I was there. I’m glad I picked the date and got it out there but in hindsight it was a little crazy to do with going to the conference then flying across the country to see family the same day. If you can time it to give yourself a bit of free time after releasing to fix the bugs, do all the social things, and monitor all the other things that would be wise.

The only real hiccup with releasing was the Google Play Console having some really awful and confusing UX. There’s no way to promote your pre-sale listing to full sale. You have to duplicate the whole store listing. The button that is labeled “Release to Production” actually means “Send the build to our review team that will take 48 hours to review before you can release unless you already did that by turning on this other setting we won’t tell you about”. So yeah, Android release was delayed…

But in the end it was released and it felt great to finally get it out there after working on it for so long!

Final Lessons Learned and Advice for Aspiring Game Developers

Everyone says this, but I’ll say it again ‘cause it’s true: Start on a small project that’s realistic to finish and build up to your bigger dream game you want to make. This was my 3rd serious attempt at making a game, but the first time I actually released one. Even still, I let my ambitions and scope go wild that made everything take way longer and could have easily led to abandoning it. There’s so many things you can’t possibly know when you’re making your first game so starting small and going start to finish will give you the experience to use in the next project.

The games industry is a very competitive space and there’s a high likelihood you won’t even make minimum wage for the time you put into it on your first game. I certainly haven’t yet. Try to find what your passion is for making games and what you want to accomplish. If you’re just in it for the money, there are easier ways to make money for your time. I’d advise seeing how far you can get doing game development as a side project to make sure you love doing it and can stick with it before pursuing it as a full time job.

If you can find people to partner with to bring a game to life that would be awesome! I think it can be a bit of a double edged sword though. You’ll have to find people that are really on the same page for what the game should be, with complimentary skills, and are realistic about committing to finishing the game given how long it’ll take to make. If the collaboration doesn’t work out it can easily kill the project.

When you’re first thinking about what game you want to make, write all your cool game ideas first. I know you have at least a couple. Then pitch the idea to your friends and family and get their reactions. Do they get the idea? Are they confused? Does it sound like fun? This should help you both refine the idea and get a sense of what it’ll be like when it comes time to promote the game. What’s the hook, and what’s the kicker? Remember that you could be working on this for quite awhile so really make sure this is the right idea that you’re super excited about. It’s a big commitment! When you’re in the thick of development it’s hard to push through all the non fun stuff like fixing bugs or just getting the sound to play correctly so you need the spark of the idea to keep you going!

Finally, don’t stop dreaming and trying. I know it can all get a little overwhelming both in learning what it takes and actually doing it, but you learn so much along the way. The journey is more important than the destination

There’s lots of interesting discussion on this post over on Hacker News too! https://news.ycombinator.com/item?id=32851196

What language am I writing again?

Wed, 23 Mar 2022 16:00:00 +0000

Have you ever had that brainfart moment when you’re writing code and forget what syntax you’re supposed to be using? An example is helpful:

This is from a recent project that used React + Typescript + GraphQL + Styled Components which is a prime combo for this sort of situation. At bare minimum we have these languages all represented in the same file.

Typescript
GraphQL
CSS
React

Depending on how you’re counting, you could also add in Javascript (since you need to know it anyways for Typescript), HTML (since the React syntax is built on it), and even the interpolated strings like ${address.street} ${address.city}, ${address.state} which have a special syntax on their own.

This isn’t particularly a new trend. Perhaps you’ve had the ‘fortune?’ in your career to remember code like this that used to be a norm.

I found this example on GitHub and stripped it down a lot for brevity and sanities sake, but even still it gives me flashbacks of seeing 4000 line PHP files that mix and match PHP, SQL, HTML, Javascript, and CSS all in one big convoluted mess.

You might be asking yourself “ok there’s a lot of languages, is that always bad though?”. One thought experiment I like to test ideas against is taking them to the extreme and see where it goes. For this example, the extremes are:

The file only has one language
The file has 10 (20?, 30?, 100?) different languages in it. You switch between them practically on a character by character basis.

Straight away I know which extreme I want to aim towards. I’m pragmatist so I’m not saying that files must only ever contain a single language, but the exercise gets you thinking about the ideal and where the breaking point is.

There’s some notable disadvantages of having many languages in a file, and particularly embedding languages within other languages like the GQL and CSS inside the strings in the first example.

More cognitive overhead
Editor tooling is harder and slower
Single responsibility principle suffers heavily

There have been some interesting experiments with composing languages together at runtime. I agree with the author’s plight that switching languages is all too hard and we end up getting locked into our choices, however I don’t agree that their solution of composing languages together as they did is the right one. As mentioned in the conclusion, there’s a lot of friction switching languages. A lot of weird edge cases. A lot of, for lack of a better term, “magic”. Well, maybe there is a better term: complexity. Since our prime directive as software engineers is to manage complexity and keep it at a minimum we have to keep looking for other solutions.

As mentioned before, I’m not about to go preaching to the world that you should only have one language per file, just that your eyebrows should instinctively move up when you see this happening. Design is all about balancing trade-offs, and this is no exception.

Have you’ve seen some egregious examples in your career? Perhaps a bunch of DSL’s mixed in? Share them in the comments!

How Hard is your Email to Say?

Sat, 01 Aug 2020 23:31:58 +0000

You’re at the doctors office, talking to an aquaintence, or ordering something on the phone and they ask the question: What’s your email? Depending on your name, age, and your life choices this can be a breeze or the dreaded question. How long does it take before you have to break out the phonetic alphabet? How many times do you have to repeat it?

Today we’re going to come up with a scoring system to measure how painful your email is to tell someone. It’s a golf scoring system with low scores being easy and each point is one unit of struggle for both you and the recipient.

Lets start with the hypothetical perfect example: bob@gmail.com coming in with a score of 0. It’s short, contains only a common name, and is on a popular mail host. From there, lets see all the ways yours is harder to say.

Non popular email domain

If’s it’s not @gmail.com, @outlook.com, @hotmail.com, @yahoo.com, @icloud.com, or @aol.com take a point. These are services people are familiar with and don’t have to think about typing. They just might judge you on which on you use.

Domain with subdomain

If you have an email from school hanging around you’re likely to be guilty of this one like I am. My school address got the alumni subdomain added to it. I’ve seen a bunch of other emails from schools with the department included @cs.xyz.edu, or other fun stuff just ‘cause like @mail.xyz.edu. Add a point for each level of domain past the normal two levels.

Non common top level domain

If you’re extra special you’ll have a TLD like .is, .io, .rocks or some other novelty. Take a point if it’s not .com, .org, .net, .edu, or .gov.

Long Username

After a certain amount of letters in the username you’re just going to have to slow down and repeat that. +1 point for every 2 characters over 6. bobsondugnutt@gmail.com gets a 3 for example.

Punctuation

Periods are pretty common in email usernames because we’re used to saying “dot” for the domain anyways but they’ll still slow down whoever is typing it and likely miss what comes after it. Any other punctuation is a great typo opportunity. +1 for any period in the username, +2 for any other punctuation in the username or domain.

Numbers

Just like punctuation, numbers are in that more unfamiliar part of the keyboard where fingers get lost and push the wrong buttons. +1 for each number.

Any hard to say words that aren’t your name

Bit of a loose definition here but you should know it when you see it. If you have a username that’s not your name (where they might be able to refernce spelling) and is not a common and easily spellable word take a point. xThreadRiPPerX@gmail.com I’m talking about you whoever you are.

Similar sounding letters

This is one I know all too well from one of my emails where I have to break out the phonetics after the 3^rd repetition and they’re still confused. “mn”? “mm”? “nm”? Add +1 for each pair of similar sounding letters that aren’t in an easily pronouncable and spellable word. Some common pairs include: BV, DG, GZ, MN, JK

Bonus: Unicode & Emoji

Good luck getting someone to type that over the phone. +5 for each unicode character.

The email I’ve been using the most over the past few years is guilty of a few of these and comes in at a score of 3 so I’m well aware of the struggle.

Let me know in the comments what score yours is and the struggles you’ve had!

You can also check out some of the discussion over on Hacker News https://news.ycombinator.com/item?id=27454467

Your Reach is More Than You Think

Tue, 30 Jun 2020 04:15:00 +0000

I’ve been in the process of moving over to a new site analytics provider and when my previous couple of posts got way more traffic than I was expecting it was pretty interesting to compare the numbers between them. I’m going to avoid naming them but you should be able to figure them out without much trouble. Analytics provider “A” is a fairly typical one, gets blocked by most adblockers, and respects the do not track browser setting even though no personally identifiable info is being stored or tracked across sessions. Analytics provider “B” is a lightweight alternative that doesn’t get blocked by most adblockers and doesn’t look at the DNT setting as far as I know.

After seeing around a 25% - 35% increase in traffic just from looking at a different analytics provider it got me thinking about all the other ways you might underestimate your reach!

Taking the analytics one step further I decided to look at the server logs and see how many more hits I was getting from the people who were able to block analytics B (or who have javascript disabled altogether). Again I saw around a 30% increase in traffic, putting the total increase around 65% over the “naive” number!

These multiplicative effects are all over the place when you create content. Comparing the number of upvotes the posts got on Hacker News to how many views they got is sitting around a 95x multiplier. Comparing reputation on Stack Overflow to the number of people reached that can be found on users pages (across a wide spectrum of reputations) is around an 80x multiplier. Comparing GitHub stars to NPM weekly downloads is a crazy 800+ multiplier. I’d love to get more examples from other sites like social networks but they keep their viewership data under lock and key for the most part. I bet they have some big multipliers too though.

Even past the single view or consumption of something you create, think of all the other ways you have more reach. In audio and visual mediums there could be multiple people consuming from one hit. How many times has it been shared in conversation? Have you ever influenced a decision or someone else’s creativity based on what you create?

The existence of the 1% rule also means that the competition for being an I N F L U E N C E R is so tiny compared to the potential audience you have when you create content and put yourself out there.

So to wrap it up, there’s so many ways you have an impact beyond the likes, upvotes, pageviews, stars, and other things we like to focus on and I wholeheartedly encourage you to get out there and make the world a better place through your creations.

A Critique of React Hooks Addendum

Thu, 07 May 2020 17:16:00 +0000

This is a follow up to A Critique of React Hooks which blew up beyond expectations and generated a ton of discussion that I enjoyed reading. If you haven’t read it yet please take the detour before spoiling the quiz for yourself! I’m not one to harp on a subject repeatedly but after seeing all the quiz results come in I thought it would be valuable to share them as well as a couple of thoughts across common themes in the discussion.

Quiz Results

With over 2,600 responses at the time of writing this is the biggest sample size of an unscientific quiz I’ve ever given! Even ignoring the last question which was mostly there for laughs, the results were still very interesting. The median score is 1 out of 4 questions despite 3/4^ths of the questions having a correct popular vote. To me this indicates that people are familiar with a few of the hooks but haven’t used all the ones included in the quiz.

Quesiton 1

The first and simplest question had the highest percentage of correct answers to no surprise. However, over 40% of you were surprised that the Child’s useEffect fires before the Parent’s after rendering both components.

Question 2

This one surprised me when I first ran it but makes sense when you take a minute to think about it. Both useLayoutEffect’s run before the useEffect’s and they run from the bottom of the component tree upwards. Hopefully you’ll never get into a situation where this particular ordering matters though since only 30% were able to predict the order at first glance.

Question 3

This question has the intentional bug of creating a new object every rerender and passing it to the child as a prop which causes the memo to run every rerender which a third of you caught. It would be interesting to compare these results with a question around a similar bug of creating a function inside the render and passing it to the child which causes the Child to rerender every time the Parent does:

function Parent(props) {
  function doSomething() {
    // … props …
  }

  return <Child onSomething={doSomething} />
}

This is fixed with a useCallback hook, and the quiz problem is fixed by either updating the object via useRef or refactoring the props of the child to not take an object as a prop. Both of these scenarios can come from refactoring a class component where you either have a class method or property that don’t get recreated every render. When converting you have to keep this in mind and take the obfuscation penalty wrapping them with hooks.

Question 4

This is the opposite sort of bug where a Child component needs to be rerendered but isn’t. It happens when useRef is needed to maintain a stable reference over rerenders but then a child component is added that needs the value. If the whole ref is passed as a prop instead of the .current value the Child component won’t be rerendered because the ref hasn’t changed. In isolation this shouldn’t be too hard to spot but can be tricky when there are many props being passed from different places to a component.

A few more thoughts

On More Stuff to Learn

There were a few comments about how easy hooks are to learn. While I would agree that reading the docs and getting the gist of how hooks work doesn’t take too long, fully understanding their ramifications, and the bugs you’ll run into, will be an ongoing process. The quiz results show that even if you got everything right, you’ll likely be working with someone that didn’t.

Adding more to learn just raises the barrier to entry. And every developer should be trying to remove as many barriers to entry because it’s in our best interest to do so.

They Don’t Compose with Themselves

Programming languages are powerful because they give you building blocks that you can compose together in many different ways. If you have the blocks loop and function, all the possible combinations are valid including nesting. loop in loop, loop in function, function in loop, etc. Hooks don’t have this property. You can’t use a hook inside a hook (excluding custom hooks that are just wrapper functions for the native hooks). You can’t use a hook in a loop. So when you go from having one thing handled with useRef to needing multiple things handled by useRef you can’t just stick it in a loop or array like you would with any other language building block. You have to redesign it to work with hooks.

Why Control Flow Matters

When you inevitably get to debugging, you should start with a mental model of what you expect to happen including all the assumptions about what state you’ll be in throughout the process. Whenever you step or log your way through and one of your assumptions is broken, you know you’ve found a piece of the bug. Now you just have to figure out why your assumption was wrong.

Between the hook execution order and figuring out why a component rerendered it can be very difficult to create that mental model of what’s supposed to happen.

Last Words

Whether you like hooks or not, I hope through these posts you’ve learned something even if it isn’t as general as I’d like :)

As always, Thanks for reading and discussing!

Lots of comments also posted over on Hacker News https://news.ycombinator.com/item?id=23102639

A Critique of React Hooks

Mon, 27 Apr 2020 18:04:00 +0000

I want to preface this critique by saying I think that hooks are not all bad. If I were starting a new react project today I would still use them despite all these flaws. However, that doesn’t make them immune to criticism. Given their design, I think there are a number of factors limiting their longevity and I won’t be surprised if Facebook comes out with The Next Greatest Thing™ to address some of their shortcomings.

1. More Stuff to Learn

The first and easiest gripe with hooks is simply that they’re another thing to learn. I was fortunate to get into frontend development at a simpler time where you could just dump jQuá on the page and call it a day. Then as new complications came along I could learn them incrementally. Nowadays there is just so much to learn. If I was a new developer and saw that chart I would want to go home and rethink my life.

Learning new things is good though! We should be learning all the time! The problem with learning about hooks is that they’re not generally applicable knowledge about computing or programming. They’re a React-specific thing. In 5-10 years when you’re programming something completely different your knowledge of hooks will be completely obsolete. Whenever possible, you should pick learning about generally applicable thing A over very specific thing B so that your knowledge can compound and pay dividends.

2. They Don’t Interoperate With Class Components

If you’re starting a new project that doesn’t use class components you can GOTO 4, but read on if you’re in the camp of working on an app from the time before hooks.

Say you’re working in a project that’s been around for a while and it’s 50% FunctionComponent’s and 50% class components. Now it’s time to implement a new feature which you think would be a perfect use of a custom hook to do the heavy lifting. This works great in the original FunctionComponent that uses it, but now you need to use that feature in a class component. What do you do?

There are a few options, but they all have drawbacks. First, you could reimplement the feature without hooks but I hope you don’t do that. Alternatively, you could create a HOC to wrap your component and pass down the hooks functionality. This creates a lot of boilerplate and doesn’t compose well though. The only sustainable option is to suck it up and convert that class component to a FunctionComponent. Depending on how much stuff is going on in that component, this can take a while and there’s a non-trivial chance of introducing a bug.

This is a problem that lessens in time as more of your app is converted of course, but can be a real challenge to being productive when each time you touch a component you have to refactor it.

3. Ecosystem Challenges

The React docs claim hooks are “Completely opt-in” and “you don’t have to learn or use Hooks right now if you don’t want to”. This seems a little disingenuous to me. Code doesn’t change but the world around it does.

As a library author, you’re forced to make some choices in the now fractured ecosystem. Do you add support for hooks? Do you also maintain support for class components? For how long? This is a hard question to answer, especially if the library leaned heavily into HOC’s.

As a library consumer, what happens when you need to upgrade a library but it made the choice to only support hooks? As the ecosystem evolves and adopts hooks you’ll be coerced more and more into using them. This is another argument for reducing your dependencies but that’s a story for another day.

4. The Rules of Hooks Limit Your Design

I’m assuming anyone who’s read this far knows about the Rules of Hooks. The reason for the rules makes sense given that React uses call order to maintain state but this constraint can really limit how you organize and optimize your code.

In a recent example, I was building a hook that returns a lookup table where many of the values in the table were hooks themselves.

  function useLookupTable(param){
    return {
      a: {
        default: useADefault(param)
      },
      b: {
        default: useBDefault()
      },
      . . . many more entries
    }
  }

To avoid returning a big new object every time this hook is called I wanted to memoize the value. Lets try wrapping it in a useMemo:

  function useLookupTable(param){
    return useMemo(() => {
      return {
        a: {
          default: useADefault(param)
        },
        b: {
          default: useBDefault()
        },
        . . .
      }
    })
  }

Oops. React Hook “useADefault” cannot be called inside a callback. So let’s pull all the hooks out and include them as dependencies:

  function useLookupTable(param){
    const aDefault = useADefault(param);
    const bDefault = useBDefault();
    . . .

    return useMemo(() => {
      return {
        a: {
          default: aDefault
        },
        b: {
          default: bDefault
        },
        . . .
      }
    }, [aDefault, bDefault, . . . ])
  }

We’ve now got rid of the error (and introduced an annoying number of temporary variables) but we’re still returning a new table every time. Why? Oh, it’s because one of the values is a new array which fails the dependency equality check every time. Enforcing each of these useXDefault hooks to return a stable value (like with a useRef) seems like a bad call since it would overly complicate them, and even if I did, as the table grows (and if each entry has multiple hooks) the dependency array soon gets out of control.

In this case I think the best solution is to refactor all these hooks to not be hooks and just regular functions that take their dependencies explicitly so that you can call them inline like in the second example and the dependency array for useMemo is manageable.

These sorts of battles with how you have to organize your code in a world of hooks is common, and it can take a lot of iteration, thought, and time to come up with a design that feels good.

5. They Complicate Control Flow

Using a few useState hooks is pretty easy to understand, but once you start needing to use all the other kinds of hooks up and down the component tree it becomes very difficult to follow the execution order of your code. How well can you reason about code if you don’t understand the order it’s executing in?

Here’s a little quiz to see how well you can follow some toy examples.

Loading…

If you aced it, congrats! I’m guessing you either ran them yourself or you had to sit back and scratch your head a bit. These are certainly contrived examples, but we all should be able to agree that simpler and less magical code is easier to work with than the alternative.

In Conclusion

I’m not asking you to never use hooks or remove them from your project. As I said in the intro, I would still use hooks in a new project, and I think they do improve the prior situation of HOC’s creating component hierarchies deeper than the Mariana Trench. But I also think they’re not the “Computing Promised Land”. It might be a while before we get there though.

So until then, just use them judiciously.

P.S. Library authors, please try to keep things simple!

Cross posted to https://medium.com/indigoag-eng/a-critique-of-react-hooks-6de10e8f14e1

Quiz results and a few more thoughts in the addendum!

Lots of comments also posted over on Hacker News! https://news.ycombinator.com/item?id=22995928

Unity Save Game Editor

Mon, 18 Nov 2019 07:58:08 +0000

In game development there's a lot of scratching your own itch and my latest diversion is a great example of that.

More often than I care to count, when I’m working on a new feature for Macrocosm something gets in a weird state and I need to see what’s going on with the save. Once I figure out what happened and attempt a fix, it would be super nice to be able to then edit the save game and put it back to a correct state to test from.

After thinking about it enough I decided to take the time from main game development and make this tool that should speed up future dev time.

Enter the Unity Save Game Editor.

Once you set your save game path & extension and choose which save file you want to work on, the editor will display a tree of all the loaded data you can then dig into and edit!

On each row there are contextual controls for resetting values, and adding or removing items from a collection. While there are a few more things I wish it had, it suits my needs for now and I think it’s in a good state to give it to the world and hopefully have other people help contribute to it to.

This will work if your save game is simply serializing a data structure, something like this:

public void SaveGame(){
    BinaryFormatter bf = new BinaryFormatter();
    FileStream file = File.Create(savePath);
    bf.Serialize(file, gameData);
    file.Close();
}

And on that note, go check out the project on GitHub!

https://github.com/dshook/UnitySaveEditor

Once you have it copied into your project in an editor folder (in my case: Assets/Editor/SaveEditor) you should see the menu item to open it in Unity’s window menu:

Hex Map Area Borders

Sun, 16 Jun 2019 04:00:00 +0000

Today we’re going to learn how to create a nice border around the perimeter of an area in a hex map using Unity and a LineRenderer. Here I have it as a dotted purple line but you can make it look like anything you want with the flexibility of the line renderer!

The foundation of the hex code here is adapted from the Hex Map Series by Catlike Coding which has been super helpful for getting started. So if there’s any bits of code I missed here it’s likely from there, but please let me know in the comments if you’re getting stuck or confused anywhere.

First up we’ll need a little background code for the data we’ll be operating on for the rest of this article.

1. Find the Area

The first step is finding the set of cells that we’ll use to find the perimeter around. For my implementation I started with a variation on a flood fill algorithm that is also close to a Voronoi diagram. There’s a great article here about it as well as many more great articles about hex maps, pathfinding and much more.

Since my city areas are changing infrequently I update them at startup and whenever one of the cities grows in influence (which affects their max radius of influence).

2. Find the perimeter of the area

Now that we have a set of cells for an area we need to find the cells on the perimeter of the area so that we can then find the points along edges of the cells for the line to go.

This works by finding the top right-most cell and then marches around the perimeter by prioritizing the direction to travel in. For example, if our last move was to the cell south east of us our priorities would look like this:

This priority ordering will ensure we stay along the outside edge of the area.

We also need to include the 5th priority as going back to where we came from in the case where there is a peninsula of cells that stick out from the rest. In that case our perimeter will contain those cells twice as the algorithm marches out and back, but this is what we want so that the line can follow around the outside.

3. Find the points for the line

And now for the fun part: finding all the world coordinates for our line renderer. We’ll do this by iterating over the perimeter cells keeping track of which direction we came from and which direction we’re going. These directions will let us determine which kind of “bend” we’re dealing with. This bend type will then determine how many corners of the current hex cell we’ll need to add to our line before moving on.

And that’s the gist of it! All that’s left is the function to tie it all together:

When you set up your LineRenderer just make sure the Use World Space and Loop boxes are checked.

Hope you learned something and saved some time!