A KPI for Understanding Your KPIs

Datinuum Newsletter - December 18th, 2023

Data Unfiltered

Track the Right KPIs

Key performance indicators (KPIs) are essential for all businesses to understand the health of their business. They also are a great benchmark when comparing yourself to the market and others in your industry.

While KPIs are important for businesses, they’re easy to develop, and most people track the wrong ones and track too many of them.

A “wrong” KPI is one that:

1. Doesn’t measure an outcome.

2. Changes too frequently.

Not Measuring Outcomes

The most common “wrong” KPI I see is tracking the number of dashboards, data models, databases, or any data solution produced. It seems like the right thing to follow—you and your company are trying to understand, use, and leverage your data to their fullest potential. However, this is the easiest way to generate a plethora of tech debt, as this KPI has no beneficial outcome for the organization.

You can have 100 dashboards, 100 data models, and 100 databases. Yet, if only 1 Excel report is used and everything else is ignored, you’re wasting many people’s time and a lot of company money.

The conversation must shift from “how” to “why” in these scenarios.

• Why are we tracking these numbers?

• Why are these data important to us?

• Why do we have these data tools?

More effective KPIs would be:

• Daily active users per dashboard

• Type I / Type II error percentage by data model

• Database performance and usage across the enterprise

These KPIs track the quality of the data work that is developed rather than the quantity, and gluttony of data junk that exists within the company.

Changes Too Frequently

Most companies and departments act reactively and need a new yearly KPI to measure progress. The goals are about remediating the issues and shortcomings from what happened last year instead of focusing on a forward-looking plan.

You can’t measure the effectiveness of any data solution implementation or data strategy if you’re constantly moving the goal post. You can respect and study the past, but you will crash moving forward if you’re always looking back.

Instead, the most critical KPIs at a company should be everlasting across the lifespan of the company.

Some may ask, “Well, what about when new technology is developed, like generative AI or blockchain.”

See above — your KPIs should never be reactive and should not measure quantity.

It doesn’t matter if generative AI or blockchain comes along because you’re success will not be measured by how many generative AI models or NFTs you produce.

Instead, you should measure KPIs around:

• Subscriber churn

• Employee satisfaction

• Customer engagement

• Workplace productivity

If you enable those things through new innovative technologies like generative AI, great, but you won’t have reactionary KPIs that you can’t track over time. You will instead have a longitudinal understanding of the success of your business.

Data in the World

SLMs the new LLMs?

Microsoft announced the release of Phi-2, a new SLM (small language model) trained on 2.7B parameters, roughly 0.2% of the size of GPT-4.

While the model is limited in its capabilities compared to GPT-4, SLM architectures will help alleviate some of the capacity concerns with LLMs until better hardware is developed.

Even when better hardware is designed, SLMs will have a place in the generative AI ecosystem.

OpenAI Guiding Your Prompts

OpenAI released a prompt engineering guide this week—which immediately resulted in someone making a PromptGPT…about prompting.

The release from OpenAI and the swiftly developed GPT highlight what I’ve always thought—prompt engineering will be a field displaced quickly as generative AI becomes more predictive and personalized based on user history.

Data Histories

The Quadratic Formula and Quadratic Equations

Quadratic equations surround us in real-life examples ranging from the flight path of a soccer ball kicked to the square footage of a new house.

For most, one of the earliest first steps in learning quadratic equations is learning the quadratic formula in algebra.

Most know of the formula, yet few know where it originated from or who created the equation.

The formula is designed to find two solutions to the quadratic equation called roots.

While there are early findings of historical societies such as Egypt, China, Babylonia, using rudimentary methods that resemble quadratic equations, the man likely responsible for the quadratic equation as we know it today is Simon Stevin.

Stevin, a Flemish mathematician, popularized the use of decimal numbers in mathematics, and his paper L’arithmétique highlighted methods that can be used to approximate algebraic equations. These methods called for the inclusion of all numbers (irrational, square root, negative) should be included as valid numbers, which later became the basis for what we know today as real numbers. Notably, he excluded imaginary numbers on purpose to hinder research.

His contributions to mathematics and data spanned trigonometry, architecture, algebra, and navigation, but his most notable contribution regarded his work with gravity.

Stevin’s use of decimal numbers helped refute earlier findings from Aristotle regarding falling objects. To disprove Aristotle, Stevin and a colleague, Jan Cornets de Groot ran gravity experiments at the Delft tower in 1586. However, due to poor instruments and a lack of proper data collection, the Delft experiments didn’t gain the same notoriety as Gallileo’s more notorious Pisa experiments three years later.

Datinuumber of the Week: 61

61 subscribers. Thank you all.

I had been ideating on this newsletter for a few months, and after publishing the first edition last week, I am grateful for the engagement and support thus far.

The learning journey and diving into the Datinuum has only just begun.

Thank You

Thank you for subscribing and reading this week’s newsletter.

If you enjoyed the newsletter, the best way to help is by sharing it with colleagues and friends.

If you prefer to listen to the newsletter, the Datinuum Podletter will be released weekly on Apple, Spotify, or your favorite podcast player.

The video podcast will also be released weekly on YouTube.

Feedback is a gift. Please reach out to [email protected] with any feedback or questions.