So What Exactly is MVP Anyways?

It’s been a little while since the last project update! Life happens and I also need to get better at switching to a “Build Share Build Share” mode instead of just “Build Build Build Build” mode which is my default. So here’s what’s happened since last time as well as a thought exercise on what exactly you need for a Minimum Viable Product.

Progress#

From last time:

My goal for the next update is to get the frontend interacting with the backend to read and set both site votes and categories. Should be easy enough but there’s a lot of stuff I glossed over on both the frontend and backend that I’m sure I’ll be refining as I go.

Goal: Smashed.

But that might be because it’s been two and half months since the last update 😅. But a lot has been accomplished in the limited amount of time I’ve been able to dedicate.

First off, I’ve settled on a name for the project: Cat Search! It’s short for “Category Search” which is what this is all about. It lets you categorize all your search engine results into whatever you want to take control of the results you see and how you mentally rank them yourself. I’ve been toying around with that name in my head for awhile now and since nothing better came to me it stuck. It’s a great name too since it’s easy to say, easy to remember, and easy to make a cute mascot for :) Even though I’m really a dog person, as you can probably tell from my profile picture, cats are great too.

Without further ado here’s what the current state of Cat Search looks like.

I got support for both Google and Duck Duck Go working so whenever you search for something the extension parses all the link results, sends them to my API for categorization, and then displays the categorized results above the main search results. All in about ~20ms right now. Granted that’s hitting the API on localhost with less data than a production environment will have but I’m going to be keeping a close eye on performance the whole time since the results are taking over the top of the page and any extra latency will be much worse for user experience since the whole layout will shift. Right now at this speed you hardly even notice the loading delay as the whole page is loading in. I’m sure there will be more work needed around optimizing the layout shift as the page loads in the future though.

Undoubtedly there will be some uncategorized results this early in the project before many people are using the extension. I’ve seeded some categories based on a few lists I’ve found but the internet is a large place and I’m only going to have a tiny slice categorized when Cat Search launches. So that’s where all the users come in to either drag an uncategorized result into a category that’s already on the page, or create a new category. All the categories you create will just be for you to start but the idea is that there will be a process looking across many users that have categorized a site the same way and “promote” the category so that everyone sees it. This will be a manual process to start with while I figure out what the right criteria is for automatic promotion.

The last feature I worked on before this post was to automatically load the next pages of results. This will likely be a paid feature since it’s a nice bonus to the core functionality. I, like you, am lazy and almost never click next page results. But I know there’s many times where a better answer to what I’m looking for is buried beneath the SEO spam on page 2, 3, 4, etc. and if it was automatically categorized into a category I highly value (like Indie Blogs) I would click on those results much more often than I do now.

And finally, I created a design mockup for the UI! It’s a little rudimentary as far as full designs go, but it’s enough to get me going. I’m sure you can tell the actual current state look is, well… very programmer art ish. It’s functional but not pretty, and that’s just because I hadn’t spent much time on visuals since I was still in the process of figuring out what this thing will become and how it will work. So next up is just snapping my fingers and making it look like the mockup!

Challenges#

In this chunk of work there certainly were a number of challenging things that took longer than I expected and some open questions to solve later.

Result Sorting#

The first was how voting would interact with dragging a result to a certain position in the category list when it comes to sorting the list the next time (or on page reload). There’s a number of things to sort results by:

• My category rank (if I’ve previously dragged a result in the list)
• My category votes (using the up and downvote arrows)
• Other people’s votes on the site and domain
• The search result order

That’s a lot of info to take into account while also making the user experience simple, make sense, and consistent. Currently I’m using the following logic which has worked well for me while using it but I’m curious to see if I get feedback on this.

• Dragging a result into the top 3 of a category counts as an upvote, anything below that counts as a downvote
• Find all the results in a category that don’t have a category rank, then sort by:
  • Combined vote counts for mine and other people’s votes
  • Tie break with the search result page rank
• Insert all the results that do have a category rank into the appropriate place in the list
  • Tie break with the combined vote counts

This logic ensures if you drag a result to a certain position it stays there, and otherwise highly voted on results bubble to the top, and we fall back to the search engine ranking.

DB & API Design#

For the API design I wanted to make sure all the necessary changes to the DB when a result is moved categories could happen in one transaction both to keep the DB consistent but also remove the complexity from the frontend and make the API calls efficient. There’s actually quite a number of DB updates that need to happen for a simple drag action:

1. Check to see if the user has already voted for this site so we can calculate the vote changes if it went from an upvote to downvote.
2. Upsert the user site data with the votes
3. Upsert the shared category data with the vote changes
4. If we’re not categorizing the whole domain, also make sure to upsert the shared domain category data with the vote changes
5. Upsert the user category data with the new category and category rank data
6. If the user is moving the result (vs copying), delete the old user category record

Figuring out the table shapes and API actions for all the combinations of (new || update), (site || whole domain), and (move || copy) definitely took some time to figure out and required some notebook work to help break it down.

URL’s#

Currently my URL matching is limited to either a domain (including the subdomains) or the URL and path (excluding any query parameters). I’d like to be able to support matching partial URL’s with wild cards for cases where you want to categorize something like a whole subreddit https://reddit.com/r/pics/*. This would also allow me to seed more category data based on existing ublacklist subscriptions. Those lists are all full regular expressions though and that sounds way too slow to allow when querying and opens up a whole can of worms. But for now I’m going to kick that can down the road to post MVP anyways.

What’s The Bare Minimum to Ship?#

Back to the thought experiment in the title I hope you’ve been thinking about as you read through all the updates though. What constitutes an MVP? How minimalistic do you really need to go? What differentiates a “product” from a demo or a toy?

There’s a case to be made for rushing to deploy your app as soon as possible and get folks in the door with only the roughest functionality. That doesn’t sit right with me though for a couple reasons. You’re giving the people that care about your product the most the worst possible user experience which will inevitably taint their first impression of the product no matter how excited they are for it. People can only look past so many bugs before they look for alternatives. If you’ve really shipped as soon as possible that also means you don’t have any way to monitor issues or receive feedback, both of which are absolutely critical for molding software into a delightful user experience that solves a real problem.

This is one of the lessons I learned from creating Macrocosm. You only get one shot at launching. Make it count.

So the big things on my list I want to tackle before it’s really minimally viable in my mind are:

• Implementing the new design
• Proper authentication (I want to allow anonymous usage for a little while to give people the best onboarding experience to try it out, but then require a login to prevent abuse)
• Recording logging for FE & BE, and telemetry for API response times
• Actually deploying the API (I want to find benchmarks for different cloud providers or run a load test myself to see what gives the best latency)
• Setting up marketing site with info and links

You may or may not notice that paid features aren’t on that list. This isn’t a charity app and I will have some forms of monetization built in but after debating myself for a bit I decided to have this be the first feature post MVP. Getting the first few people using the extension, starting to categorize the web, and making sure it’s working well are all more important things than the pro features to start with.

Hopefully I’ll have at least one more update for you before I get there!