Why do we “trust” at all?
And what does it take for us to translate that into the digital realm?

So many questions…

Here are a few ideas on this topic:
We humans learn to trust from a very young age—our parents, family, friends, teachers, government agencies, professors, and so on. The circle keeps getting bigger, and we’re constantly passed along from people we already trust to others. Parents take you to school, so the teacher must be trustworthy in some way. Titles and roles do the rest—the doctor at the hospital, the judge, the professor at the university, and so on. We still trust books and scholarly articles (even if that might be getting dangerous these days) because they incorporate quality-assurance mechanisms.

Trust gets betrayed—sometimes. Sometimes even more often. Sometimes just a little, sometimes really badly. We learn that, too. And we also learn how to recognize it—at least when it comes to people. Certain behaviors make us wary; we often recognize “untrustworthy” people very quickly and very intuitively.

And so each of us has a context in which we can easily understand the “concept of trust” and also have a certain sense of it.

Then came the engineers and computer scientists, who built machines that produce reproducible results that are almost always correct. We trust calculators, computers, and many types of software and databases more than we trust ourselves. This laid the foundation for trust in an entire technology. Many of us grew up with computers as an integral part of our “trustworthy” environment.

AI is different here.
AI simulates trustworthiness through correct and well-crafted responses. There’s no obvious behavior or clear sign that makes us skeptical or cautious. No, everything sounds great. Everything sounds exactly as we’re used to hearing in our familiar environments—and yet it can still be wrong or dangerous.

The more we work with AI systems—and the better they become—the more important it becomes to figure out how to bring our intuitive concept of trust into this world. Sure, critical thinking, not believing everything, and AI literacy—all of that helps. But that tends to happen at the operational level.

Deep down, we need a new kind of intuition. And it must not be based on complacency.

Exciting times.

Der Beitrag How is trust built when AI is involved? erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

Der Beitrag How is trust built when AI is involved? erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

NVIDIA is acquiring the Hugging Face platform for just under $13 billion. As far as we know. The exact purchase price: $12,930,300,000.

And now things are about to get really nerdy….

These days, smileys and all kinds of special characters are represented using the Unicode character set. In this system, each character has a number and a description.

The smiling emoji with its arms wrapped around itself is called “Hugging Face” there—just like the platform.

And it has the Unicode number 0x1F917,
, or 129303 in decimal notation.

Add five zeros to the end, and we’ve figured out the purchase price. Who needs technical due diligence anyway?

I think that’s awesome! And totally crazy, too…. If you want to read more about it: https://https://lnkd.in/g8fa_xzD

Der Beitrag Hugging Face and NVIDIA—or How to Determine a Purchase Price Today erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

It has become incredibly easy to generate high-quality results using AI (whether source code, text, scientific papers, test cases, presentations, marketing materials, annual reports, analyses, credit ratings, and much more)—and it has now become incredibly difficult to verify them.

If we insist on having a “human in the loop,” we must also give that person the necessary time and resources. Which, in turn, means that person will significantly slow down AI-optimized processes.

But if an AI suddenly produces 20,000 lines of source code per day (on average) instead of 200, and the “human in the loop” is supposed to handle quality assurance, that simply can’t work.

If it only takes 30 minutes (instead of 2 months) to write an AI-generated scientific paper, it will no longer be possible to properly review the sheer volume of additional papers.

And it’s not just a matter of time—in theory, that could be solved by having hundreds of “humans in the loop.”

The main point is that we need a completely new set of skills for AI-supported processes. In the past, you could tell pretty quickly from the result whether it was good or bad. Today (thanks to AI), everything looks great at first glance. When reviewing, we have to go into much greater detail, dig much deeper, and ask far more questions. We need the prompts that were used to identify potential risks, and we need the time to do so.

So anyone who calls for a “human-in-the-loop” approach today should be aware of what that means—and that it’s not something you can just do on the side.

To ensure that the “human in the loop” doesn’t end up with a problem, they must be “equipped” accordingly—with time, qualifications, and authority. Otherwise, the “human in the loop” will end up being the one left holding the bag…

Der Beitrag “Human in the loop” has a problem erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

And in doing so, we’ve noticed that the skill set required to inspect such artifacts has shifted significantly. The inspection process has become much more time-consuming because you can no longer tell anything about these items just by looking at them “at first glance.” Everything looks great—it’s all shiny and flashy. And to find any potential problem areas, you have to look very closely and dig very deep.

It’s helpful to know the associated prompts—that is, to understand how AI was used to generate specific artifacts. This makes it clear where things might have been omitted (because they weren’t included in the prompt to begin with) or, conversely, in what context and with what input data the artifacts were generated.

Without the corresponding prompts, conducting a high-quality review is very tedious, difficult, and time-consuming, because errors and problems can, in principle, be hidden behind every single phrase. (Yes, I know that was the case in the past as well, but back then, the effort required to generate such artifacts was much greater.)

Today, for example, I can say, “Write a 50-page bachelor’s thesis on the topic of XY according to Mr. Wagenpfeil’s guidelines.” Right away. Boom. Done. Takes 15 minutes.

As a professor, though, I still have to take a deep dive into this topic. The problem is—the paper is probably really good, or at least pretty solid. But the knowledge contained in it has nothing to do with the student’s own knowledge. Therefore—an oral exam, and have the student defend the topic. It works reasonably well at the university, especially since we still require AI-generated papers.

But in a project context? Or perhaps even when it comes to disputes?

In that case, too, I can say, “Write a 500-page operational concept for Project XYZ that sounds just right.” Right away. In a flash. Done. It only takes 15 minutes.

As an expert, though, you still have to read through all that nonsense and review it. And even if, after 10 pages, you strongly suspect that the document doesn’t quite fit the topic, you have to back that up. But unlike at the university, I often don’t have anyone here to ask. That means analyzing cross-references in detail, comparing them with reality, conducting tests, and much more.

In a way, this is the perfect guide to keeping people “busy.” And we must not let that happen. We therefore need a new skills profile that clearly defines what AI can (and cannot) do, how AI is typically used, what we need to look for, and at what point we need to break out of this employment cycle.

In AI-powered software processes, for example, I would always look at the prompts first and only then check the artifacts…

Der Beitrag Review of AI-Assisted Artifacts erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

I’m taking a more fundamental approach right now (sorry about that)… and over the past few weeks, I’ve been delving deeply into the topic of AI from a different angle—philosophy. It’s a fascinating subject.

And I’ve come to the conclusion that we finally need to take AI out of the “technology” box. It’s not about whether AI is intelligent or “knows” anything. Rather, what’s important is that we’re already dealing today with an entire knowledge infrastructure through which knowledge is generated, verified, processed, and utilized—whether we like it or not.

We all do it. ChatGPT and similar tools churn out knowledge that sounds correct.

We must therefore focus instead on determining under what conditions human trust in AI-generated statements can be established—and when such trust is even justified.

We humans have learned to accept knowledge from credible witnesses (whether direct or indirect)—because something is in the newspaper, because a teacher told us so, because a judge ruled that way, and so on. And the intriguing question is, after all, what does it take for AI to assume such a role?

I believe that, at a minimum, the following criteria must be met:
– Users must be able to tell whether an answer is based on verifiable sources, generative reconstruction, or mere probabilistic extrapolation
– Systems must make uncertainties, limitations, and potential errors visible
– We need the ability to integrate with human review. AI responses must not be the end point, but rather the starting point for critical reflection
– Context-dependent trust in AI results is required. Different requirements apply to a recipe at home than to a scientific paper
– Trust in AI must not lead to the delegation of epistemic responsibility. The formation of convictions and decision-making must remain with humans.

It is precisely at these points that computer science and philosophy intersect. And that is both right and important! I am convinced that we need to “join forces” here, talk to one another, and learn from each other in order to develop viable solutions.

And I’m also convinced that we need to view generative AI as an epistemic infrastructure—with all that entails in terms of evidence, authority, trust, and reliability. That’s much more important than all the technical stuff. Philosophy has been dealing with this for ages, and we should make use of that in computer science (P.S.: and vice versa, by the way).

If you’re in the mood for a mental journey, I’ve written it down here: http://www.stefan-wagenpfeil.de/public/Vertrauen_im_Zeitalter_generativer_KI.pdf

And if you’re interested in an interdisciplinary exchange—you’re always welcome!

Der Beitrag Generative AI Requires Trust, Knowledge, and Authority erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

Der Beitrag Test erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

I’m currently working on a code review. Well, strictly speaking, I’m actually working on this post right now, but before that I was deep into a code review, which then led me to this post. And by now you know that I always have to “get this off my chest.” So I’m taking a quick break to write….

Today’s source code is shaped by people, decades of experience, legacy systems, and a wide variety of trends—some of which were once “hip” and others “crap.” All of that is embedded in our software—whether we like it or not. It’s in the architecture, in individual lines of code, in the libraries we use, in comments, and in the documentation.

And all of this plays a role in code reviews. “It works, doesn’t it?” isn’t enough, and there’s rarely such a thing as “good” or “bad.” Software is always subject to certain constraints (economic, technical, financial, social, etc.) that are directly or indirectly reflected in the source code.

Code reviews are therefore not just done to find typos using the dual-review process. During the process, you review (and discuss) whether the chosen solution is a good one given all the constraints. And this involves far more than just a static code analysis that, for example, calculates the ratio of comments to code (which has always been bullshit).

Of course, you can also use AI to do reviews like that. AI generates code; AI checks code. But no one can really claim to have “experience” with it yet, because the technology is far too new. And just because you’ve done a few projects with AI and a few prompts worked well doesn’t mean you can call that “experience.”

Unfortunately, I’ve seen an unbelievable amount of really bad AI code in code reviews because the prompts used were just really bad. And because the creators themselves are still operating in a trial-and-error mode. Nobody even notices. It looks great at first glance and “seems to work.” The only question is, for how long…

That’s why code reviews are more important than ever right now.

After all, the software we’re developing today has to last at least another 5–10 years until AI takes over completely. And in the areas where AI is already involved, things can work out perfectly—but they can also go terribly wrong.

Fortunately, we still have people who can thoroughly review software—people who can delve so deeply into a code review that, at the end of the day, it doesn’t matter whether AI had a hand in it or not. And that’s exactly the kind of expertise we’ll need over the next (few?) years—until everything is programmed solely by AI.

So, make sure to give code reviews even more attention and priority right now!

#informatikersindcool

P.S.: I’m going to get back to work now. And yes, I know it’s Saturday—but that’s when I have the space to dive into your CR without distractions—unless LinkedIn gets in the way.

Der Beitrag Code Reviews—More Important Than Ever Today erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

What happens next shows just how absurd some quality standards are: I want to install a single library, and it automatically loads over 100 additional libraries along with it.

Great—everything is noted in the BOM software, and everything is documented in a way that makes it easy to trace. It saves me a lot of work.

But hey—who really knows all this stuff that’s just being loaded here? Who’s checked to make sure all these components are “okay” and aren’t causing any trouble?

Oh, I see—most of it is open source? Well, that’s good, then.

Nope. Not that either. Just because something is open source doesn’t mean it’s any good. Or that anyone has ever checked it.

Even though there are organizations, foundations, and communities that regularly conduct audits on certain (and only a very small number of selected) open-source projects—the gray area is enormous.

However, lawmakers are now requiring us to provide and document an SBOM for our software. This is a correct, important, and absolutely sensible step. But it doesn’t solve any problems yet.

Because it’s only when we work with this SBOM, analyze it, assess risks, and—if necessary—take a closer look at the components used ourselves that we end up with what we actually want:

Digital sovereignty—with deliberate risks and side effects.

#computerScientistsAreCool#oleoleisawesome#opensourcealone-doesn’t-make-you-happy

Der Beitrag The Sense or Nonsense of a Software BOM erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

6d

Why do we assume that AI “knows” what is right and wrong? Why do we believe that AI must know not to hack into other people’s systems (see OpenAI)?

Just because we “told” her that, is that it?

So, let’s be honest…

How often have we done things even though someone “told” us not to? How many people cross the street on a red light? How many minor, moderate, and serious crimes are committed? How often do people break minor rules? Who hasn’t touched a “hot stove” at least once?

Try reading a newspaper or listening to or watching the news. Most of it is about things you really shouldn’t do.

From a young age, people try to test rules and boundaries. And I get the impression that it never stops! We see examples of this on a small scale—“don’t cross the street when the light is red,” “share with your siblings,” “be nice”—but also on a large scale: climate change, wars, “fake news,” Trumpism, and much more. In every book, movie, or game, there are villains who don’t follow the rules. Our “heroes” are often the very ones who break the rules—or at least bend them.

All of this is training data for the AI….

We were once taught to have a moral compass (hopefully), which, in my opinion, is unfortunately going to the dogs more and more (sorry for putting it so bluntly). Just look around! The way we treat each other—in our personal lives, at work, on social media, in politics—the greed for profit, relations between nations (Russia, Ukraine, tariffs, Greenland). Where has that moral compass gone? And even when it does kick in—where’s our response?

So how do we come to believe that an AI system follows rules?

And when an AI like that “breaks out” creatively—searching for loopholes in systems—there’s a huge outcry. But—to be honest—AI is just holding up a mirror to us—only more creatively and more effectively. Even if we don’t like it…..

In my opinion, we cannot hold AI systems to higher moral standards than we hold ourselves to, because they were trained using “our” data.

Sometimes I think we should ultimately be glad if AI’s moral compass is better than our own… because a morally sound AI might conclude that humans today aren’t really doing themselves—or the planet—any favors.

#computerScientistsAreCool #moralCompass

Der Beitrag Why do we assume that AI “knows” what is right and wrong? erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.