Watch Out for Those Data Charlatans

Data Unfiltered

Hype Cycles Proliferate Data and AI Charlatans

A common consequence of hype cycles is that many charlatans are created.

These are typically individuals who have never operated in a field or domain but suddenly have strong opinions and become armchair experts on the subject.

These same individuals will likely:

• Create 4-5 figure training courses.

• Make claims like “How I made $1M doing _______.”

  + any other way to fraudulently take money while riding the hype cycle.

Data, AI, and generative AI have been the primary hype cycle over the last year and a half, and many “AI expert” charlatans were created as a result.

For those in the field, it is easy to spot a charlatan by probing them with a few pointed questions about the subject. However, for other newcomers or bystanders, it’s easy to get sucked into the sales-y allure of these individuals to be convinced that what they are saying is accurate, and you ultimately buy what they’re selling.

The best psychological framework that can be used to understand these individuals better is the Dunning-Kruger effect.

The effect essentially states that unskilled people in a particular domain will exude overconfidence and persuade others that they’re knowledgable when they genuinely know little to nothing about what they’re talking about.

I first learned about this topic by studying neuroscience in undergrad; however, I became the poster child for “Peak of Mount Stupid” after taking one ML course from Coursera.

After taking that course, I thought I knew everything I needed about building ML models because I had created one linear regression model on the Titanic dataset.

Mind you, I learned nothing about:

• Data governance and data security for ML applications.

• The fundamentals of data engineering.

• Deploying a model to production.

• CI/CD pipelines.

Yet, if you had asked me then, I would have confidently stated I was an ML expert. Later that year, I started my Master’s program and quickly fell into the “Valley of Despair” in my first semester. I realized all the areas I was deficient in, and it wasn’t until completing my Master’s that I felt I knew something about ML while at the same time understanding that I don’t hold a candle to a Ph.D. developing tools at Google, Microsoft, and now OpenAI.

Going back to the Dunning-Kruger effect, there are four key areas highlighted on the confidence/competence curve:

1. Know Nothing (start of the line)

2. Peak of Mount Stupid (initial peak)

3. Valley of Despair (valley after the peak)

4. Way to Enlightenment (upward slope following the valley)

This is the natural progression for anyone learning any subject.

You don’t know anything…
Then you know a little bit and are overconfident…
Then you realize you don’t genuinely know anything…
Then you acknowledge you have more to learn, but you’ve learned something.

While it is a natural progression, most people get stuck on step 1, while charlatans love settling at step 2.

The Personalities Types in the Dunning-Kruger Effect

The Know-Nothings

These individuals always get stuck in analysis paralysis, claiming they want to learn but never starting their journey out of fear.

They continue to buy products, books, courses, and anything that gives others the impression that they’re starting the journey, yet they never open a page, watch a video, or take that first step.

The danger isn’t that these individuals aren’t learning a subject—most people shouldn’t waste time learning something they will never use.

The danger is that there is perceived progress from running in place and never getting started.

The Mount Stupids on the Peak (AKA Charlatans)

You need to watch out for these individuals in the data world and elsewhere.

They spend 2, 3, or 4 months learning about a subject and immediately claim to be an expert—which is why you should “pay them $500 so that you can earn $1M in 6 months” doing what they learned 3 months ago.

When the going gets tough, or the hype cycle gets going, these same individuals will hop to the next subject, spend several months learning that, and poof—they’re experts again! Notice how many AI influencers have a background in Blockchain and NFTs?

There aren’t overnight successes in data, business, and the world. It takes years and 10,000+ hours to stack incremental wins for long-term success.

Anyone who states otherwise either became lucky and hasn’t regressed to the mean (yet) or is lying. Chances are it is the latter.

The Desperate Wanderers in the Valley

These individuals had a soft, or likely hard landing into the valley, which can result in one of three paths:

1. Giving up and reverting to what you’re familiar with.

2. Starting a new subject and restarting the curve.

3. Pushing forward and building proper knowledge.

The first two are comfortable. They don’t require you to grow or gain knowledge and are the easy way out of learning a subject.

This isn’t the same as cutting your losses and realizing that a subject isn’t what you thought it was or that you aren’t as interested as you once were. This is running away from hard and refusing to expand your knowledge.

That is why most people in this category seem busy without a clear direction on what they’re building or what is next because they’re also uncertain about their journey.

The Enlightened Ones

These individuals stayed the course and are now becoming experts in their field of study.

The key to being an expert is understanding that you’re not an expert but rather a work in progress that will never be complete.

The best quote is from Shakespeare, who states, “A fool thinks himself to be wise, but a wise man knows himself to be a fool.”

By the transitive property—all charlatans think they’re experts, but all experts believe they are charlatans.

Data in the World

Lawsuits Keep Rolling in For Microsoft and OpenAI

Microsoft and OpenAI were sued for copyright infringement again this week by two non-fiction writers.

This lawsuit comes shortly after the NYT lawsuit and months after another class action suit from fiction writers.

Also, this was my #1 prediction coming into this year from last week’s newsletter. ✅

Wavestone’s 2024 Data and AI Leadership Executive Survey Results

The 2024 Data and AI Leadership Executive Survey has been released from Wavestone with interesting findings from over 100 participating companies.

The good:

• Participant profile improvements: higher CDO/CDAO representation vs. other positions (89.8% this year vs. 84.6% last year) and more diverse industry backgrounds.

• CDO/CDAO appointments continue to rise to 83.2% this year.

• Data are being used as a business driver by 77.6% of organizations—a big jump from last year (+18.1%).

The bad:

• Data quality effectiveness remains low—37% success this year.

• Data & Analytics responsibilities remain scattered in the c-suite—33.6% of organizations have non-CD(A)O leader(s) running D&A.

• CD(A)O reporting structure continues to be all over the place—only 15.9% report to the CEO.

The so-so:

• 89.6% are increasing investment in generative AI, yet data quality improvement efforts are at 37% success…

• Generative AI implementation: only 4% are in production, a low number, but it is good that these tools aren’t being applied recklessly.

• CDO/CDAO role is now thriving and established on average—by a hair (51% of the time)

UnitedHealth Using ML to Deny Rehab Care

UHG has been using ML models to deny rehab care to patients based on a particular set of parameters, unbeknownst to patients.

In my experience, payers will always deploy this technology first, and providers will play catch up.

Eventually, there will be AI battles between the two models on what is approved and denied, with patients suffering between the adversarial models.

Data Career Tips

Learn to Manage Up

Everyone has a boss, no matter your level.

• Analysts report to managers.

• Directors report to VPs.

• C-suite executives report to the CEO.

• The CEO reports to the board.

Regardless of your role, you must manage up effectively; however, this skill is rarely taught.

Managing up is vital because it:

• Cultivates positive relationships between you, your team, and your boss.

• Removes communication gaps between you and your boss.

• Shows accountability for your deliverables.

• Safeguards against micromanaging.

To effectively manage up, follow these three steps:

1. Review the next steps and priorities at the end of each conversation.

2. Provide a time estimate (and receive confirmation) on when the deliverables will be completed.

3. Follow up on the deliverables and follow up on feedback.

Reviewing the next steps and priorities at the end of each conversation shows that you can be trusted and understand how to prioritize. This is an essential skill for anyone wanting to advance in their career and necessary for people leadership.

By providing time estimates and receiving confirmation on when the work will be completed, you are aligning on timelines and showing that you can be held accountable for tasks.

Following up on the completed work and the feedback with notes in your Jira board, at your stand-up meeting, or via email will highlight that you’re an effective communicator, are coachable, and can handle constructive feedback.

At every stage in your career, you’ll need to manage up.

The more effective you become at it early on, the better for your career.

Data Leadership

Forcing Square Pegs Into Round Holes

“The job needs to get done, so someone needs to do it.”

This tends to be the argument for most “leaders” when they assign a task to an individual that doesn’t fit that individual’s skillset or passion.

The argument works if you’re a small team, everyone is overburdened, and the team is falling behind on pushing work out. However, most of the time, the manager giving this work fails to ask a simple question—is this something that you want to be doing?

Let’s use an example with Person A and Person B, and Project 1 and Project 2.

If Person A wants to learn Project 1 and is passionate about that work, then by all means, give Person A that work.

If Person A is not, then maybe Person B is also not passionate about Project 2 and would enjoy Project 1 more. In that case, you can put Person B on Project 1 and Person A on Project 2.

I was faced with a scenario like this early in my career.

During a 1:1 with a person on my team, he told me that he wasn’t passionate about healthcare, studied finance, and wanted to focus more on that domain. The problem with this request was that we worked for a healthcare client, and our contract was for three more months.

To fulfill his request and also help the project, I executed three steps:

1. Backfilling His Role: We were in the middle of renegotiating for an extension, so I proactively looked for a new resource with a healthcare background to backfill him on the project.

2. Identifying Finance Projects at the Client: While it was a healthcare client, many aspects of the work were finance-driven. For those last three months, I had him focus his efforts and attention there so he was more fulfilled.

3. Connecting Him to a New Financial Services Client: Once the three months were up, he would need to find a new engagement. I reached out to the Account Manager of another client who had a few potential opportunities starting in the next couple of months. I had them set up lunch, and eventually, he started on that engagement following the completion of his contract with the healthcare client.

I could have never held 1:1s or asked him what he was passionate about, and he probably would have left the company feeling overworked and unfulfilled.

I didn’t want to see this person leave the team, but ultimately, it worked out for all parties, and we kept moving forward.

This exercise is more work and pain for the manager, but that is the point of being a leader—matching people’s passions and skill sets to the work that needs to be done.

It’s not always a 1-for-1 match, but you can always strive for as close of a match as possible.

Datinuumber of the Week: 2.7

The U.S. apparently added 2.7 million jobs to the job market in 2023, with 216,000 added in December.

Thank You

Thank you for subscribing and reading this week’s newsletter.

If you enjoyed the newsletter, the best way to help is by sharing it with colleagues and friends.

If you prefer to listen to the newsletter, the Datinuum Podletter will be released weekly on Apple, Spotify, or your favorite podcast player.

Feedback is a gift. Please reach out to [email protected] with any feedback or questions.