Der Beitrag White house directly connected to Starlink… erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

Der Beitrag Bachelor Thesis – Virtual Pacemaker erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

One of the most common and most important models in information theory is the SECI model by Nonaka & Takeuchi, which describes four phases: Socialization, Externalization, Combination, and Internalization. Briefly, this means that in the socialization phase, knowledge is transferred and gained between people by talking, discussing, viewing, etc. Externalization focuses on ways to document knowledge for others: we write documents, store data in databases, put comments in collaboration platforms, add sections to wikis or the company intranet, and so on. The combination phase uses all these documents to derive new knowledge, detect common topics, categorize, or archive information. Finally, the internalization phase describes that people read documents, consume existing content and actually learn something. They internalize new knowledge.

Companies that have a good knowledge management, establish processes to maintain knowledge through all four phases by conducting internal trainings, increasing documentation efforts, use knowledge management systems, calculate knowledge quality measures and safeguard their knowledge when people leave while providing existing knowledge to new people being onboarded. They improve their knowledge by actively managing it. In many of these companies, even a Chief Knowledge Officer (CKO) at C-level is responsible – with a team, a budget, clear guidance, governance, and goals.

So, what happens, if now GenAI is employed in the process of externalization and combination?

Currently, many people tell us how GenAI can improve writing and document generation processes. Many products claim that people will be faster with GenAI. And – of course – writing documentation can be a boring and disappointing task. So, why not having ChatGPT generate some paragraphs, with some UML-diagrams or statistics, with some nice images, etc.? Users can finalize their tasks faster and focus on more thrilling jobs. In addition, ChatGPT can create nice summaries, annotations, keywords, taxonomies, can do some topic modeling, can do SEO for our articles, and much more. So what’s bad about it?

In the short term – nothing. GenAI can indeed improve a lot of tasks in the area of knowledge management. There are fancy tools available, which reduce the work effort of our highly skilled people, a lot.

But in the long run, GenAI can harm or even stop the knowledge engineering process in companies. Let’s make an example: currently, people create use-case diagrams in software documentation to outline what processes a software is planned to be capable of. People are trained and skilled to model such use-cases in UML, which results in nice pictures and (sometimes) lengthy descriptions. If these people use GenAI to generate use-case diagrams or descriptions, no problem. They can double check the generated sections, correct them if needed, and are faster and happier than before. Knowledge is still externalized – now having some generated sections in it. When new people join the project however, the onboarding process will look different: they will no longer be trained in writing use-cases or UML models, they will be more and more trained in creating appropriate prompts to have this done automatically. Pushed by the management, which is now used to faster processing times. However, such people will no longer be able to double check the results. And as all those GenAI generated sections are still externalized (and e.g., stored in Sharepoints and document databases), also the combination phase will “read” through these documents to find commons, to group knowledge, to prepare knowledge for new users, etc.

And now two things will happen:

1) GenAI content will be considered more important than traditional, human written content, as it will have a higher annotation quality or better keywords.

2) as we employ machine learning on our data to detect knowledge clusters, GenAI content will additionally be marked as being very relevant company knowledge, as the patterns of generated content will be more uniform than the patterns created by hundreds of individual employees.

This means that in the combination phase, human produced knowledge will be regarded less important than GenAI generated content. When we use this combination phase to onboard new team members, they will be onboarded based on more and more generated content from now on, and no longer with the companie’s human knowledge. Even worse, if GenAI content is wrong, biased, fake, or hallucinated – now one will be aware of it.

In the long term, human and the company’s knowlege can be lost. And as knowledge is still regarded as one of the most important assets of any company, companies might lose their knowledge foundation. Of course, this is a bit black&white painted, but in the long term of knowledge creation, preservation, and gathering, there will for sure be effects, if GenAI replaces human generated content.

So – what can we do about this?

Even today, we already have to deal with bad, wrong, misleading knowledge. That’s why we introduce knowledge management. And why we introduce quality measures. In the near future, some of such quality measures will increase, as new content will be “better” thanks to GenAI. To ensure that this is not a misleading measure, we need to introduce additional metrics focusing at the combination and internalization phase. We have to distinguish between human and GenAI content, clearly mark GenAI content and decrease its relevance for machine learning algorithms to calculate data clusters. Within companies, this can easily be achieved, but must be considered from the very beginning. Finally, we must also increase the efforts on human-to-human interaction in order to transfer knowledge via socialization. Because, this might be the most relevant source of company information in future.

People should always be more important than GenAI.

And knowledge needs to be managed more than ever!

Der Beitrag Knowledge Management in the era of GenAI erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

We all agree that somehow also digital twins need to be uniquely identified. As they should represent real-world objects (and people), they collect personal and private data in virtual environments and all this needs to be protected.

However, we can create multiple digital twins of the same real-world object or person. We can create slightly different digital twins to simulate a particular behavior. We can define which elements should be manifested in our digital twins, and which we want to hide. So… how can we identify avatars, their properties, their connection back to the real object or person behind it? What happens, if such avatars are exchanged between virtual worlds?

Answers to these questions are important for
– identity management as such
– digital manufacturing
– virtual and augmented worlds

I’m happy to share a 25% discount voucher for this year’s Identiverse, where I have a talk about these topics, the corresponding challenges and possible solutions: https://lnkd.in/eEdPb-Dr

Here’s what to expect there: https://lnkd.in/eVv3eesa.

Would be great seeing many of you in person!

Der Beitrag Identiverse 2024 – Digital Twins erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

Until GenAI came up, one of our major goals in Multimedia was Information Retrieval, i.e., the analysis of objects in Multimedia Assets, their representation and storing them in effective and efficient ways. All of that based on real-world scenes. We had to deal with photos, videos, textual descriptions, that have their origin in things that happened in the real world. Simply, because anything else was not possible (putting aside Photoshop and Hollywood movies). We came to the conclusion that multimedia represents our real world.

In addition, there were virtual worlds (Fortnite, Minecraft, Roblox, the Metaverse, and many many more). Here, it was clear that these worlds have been made by developers, UI/UI-artists, designers, and that anything we see there, is a virtual reality, i.e. it is not real.

And then came GenAI…

GenAI produces multimedia assets that look “real”. However, they aren’t. They are calculated and generated by enormously powerful AI systems. They are completely virtual! Any picture created by Midjourney is virtual. Any Video by Sora is virtual, any text generated with ChatGPT is virtual. None of this is “real”, they are just based on real-world training data and coincidentally look like something similar could also have happened in the real world.

The current GenAI systems are trained with real-world data (because no generated data existed in the first place). This means, the generated output is pretty close to the training data. However, when more and more virtual-world (i.e., generated) multimedia assets are produced and also used for the training of GenAI, the output can be arbitrary far away from “real”.

We are stunned by GenAI videos because they seem to be real. However, they are fake by nature and can show anything a user is able to describe somehow in a prompt. This is similar to the design of virtual assets in prototyping, marketing, simulation, and Virtual Reality, but much easier and faster.

As VR is also built based on Multimedia Assets, any VR element can also be created by GenAI, whole new virtual worlds can be built by prompts only. Prompters can now create their own superhero avatar performing things in their own 3D virtual world. There will be an enormous amount of multimedia assets, that show virtual scenes, none of which ever happened in the real-world. And this is what we typically call Virtual Reality.

Any GenAI content is virtual reality.

You can check out videos at Sora https://openai.com/sora and consider that any of these videos never happened in the real world.

he Sora GenAI video above shows a beautiful flower – which never existed or exists in the real world. It’s just an imagination of something that could be similar to what we call a flower in our real world. Such a video must never be used in education, because it would teach the students a false truth. Nobody would teach real physics in a Roblox world, where “people” can jump 300 meters high. Nobody would teach geography or history based on “Game of Thrones”. Because everyone knows, these are just virtual worlds, movies, imagined stories, etc.. Until now, we had a clear understanding of what is real-world, and what is virtual-world, because we were simply not able to generate and visualize other realities.

With GenAI, this is now possible, as simple as doing a Google Search. But the query “show me a video of an opening flower” does something completely different now: while Google will find real-world videos to show how a flower opens, GenAI will calculate a virtual flower and visualize how this virtual flower could open in a virtual world. If this video is then uploaded to the internet – how should another user be able to distinguish?

There has been said a lot about regulation, watermarks for GenAI-content, and so on. All this is true and mandatory. But we have to accept that until then, we arrived at a point where GenAI can build any virtual reality we want.

From a science and research perspective, we can now merge any concept known from real-world Multimedia Information Retrieval into the virtual world and vice versa. There is no longer a differentiation between real and virtual. GenAI somehow closed this gap. We need to redefine our understanding of virtual worlds, because now anyone can build them using GenAI and GenAI is VR.

Therefore, we should no longer differentiate between GenAI and VR!

Der Beitrag GenAI is the new VR erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

Fact is, that many manufacturing companies (automotive, construction, machinery, healthcare) already have digital twins (i.e., the representation of their real-world-data in digital form) in a more or less proper shape. But… what can we do with this data?

Simulation

Digital twins help to avoid high efforts in product development, and can also be used to optimize the assembly line. NVIDIA and Microsoft(https://www.nvidia.com/de-de/omniverse/digital-twins/siemens) showcase various real-life-industry applications and how they can improve both the store floor and product development. Until the tiniest bit of information (e.g., how many screws do fit into that basket?), factories can be simulated BEFORE they are actually constructed. Visualized in photorealism with VR, you can walk and test your assembly line before it is actually built.

Quality and Machine Learning

For quality aspects, it’s highly important to have good training data. If you want to detect scratches on surfaces, ML needs to have good training data. Even worse, in automotive, car sensors detecting e.g. bicycle drivers, need to have training data. Bicycles at day, night, rain, in red, black, blue, with cloudy skies, etc. Much of such training data is nowadays generated by AI, as no one on earth could recruit so many different bicycle drivers in different conditions. By doing this, millions of training-records can be produced to train the actual cameras and sensors in our cars. Another big advantage: when you generate training data, you can also generate accurate annotations and meta data for it – a task that would have to be done (or at least supervised) by humans otherwise. To validate the generated training data, RAG (Retrieval Augmented Generation) is employed to check the generated image’s content against real-world physics and shapes.

Visualization

Many vendors use AR/VR to visualize industrial content. In the area of service, 3D-BOMs are highly important and can be collaboratively rendered (see http://www.threedy.com). In the assembly line, AR can be used to highlight certain manufacturing steps and guide the engineers through their tasks. Product vendors, like e.g. PTC focus on various topics(https://www.ptc.com/en/blogs/corporate/what-is-industrial-metaverse) and such solutions are already production ready and in place. In the area of construction, simulations of the final product help to improve the quality and decreas the time-to-market.

Infrastructure

Of course, in a connected world, seamless and fast access to any information is required. Being it sensor data from machines transferred back to the factory, or service information at a customer, it is important to have fast, reliable, scalable bandwidth. It was amazing to see what e.g. Nokia is delivering here(https://www.nokia.com/metaverse/six-metaverse-use-cases-for-businesses/). One slogan that hit me was “6G on the moon”, not meant as a joke. Our industrial applications need to be connected wherever they are and if mankind starts placing industry components to the moon or even to other planets, connectivity is an enormous challenge – but: there are already solutions in place, which was quite amazing.

Data and Standards

This is a topic where we are just at the beginning. Currently, many vendors use their own formats, quasi-standards, and ontologies to describe the semantics of their 3d models. However, based on the Open-USD (Universal Scene Description) format, an alliance of the big players (Autodesk, Apple, NVIDIA, Pixar, Adobe, https://aousd.org) has been founded to collaborate on industry solutions.

Regulations and Protection

Finally, also the governments (e.g., Germany: https://www.plattform-i40.de/IP/Redaktion/DE/Downloads/Publikation/Industrial_Metaverse.pdf?__blob=publicationFile&v=2) and the European Union (particularly, the AR/VR Coalition) provide and prepare a set of legal requirements and suggestions about how to deal with data and services in the area of Industrial applications. It is important to understand the boundaries and to check what can be used without security constraints. Companies already train their models on verified and copyright safe data to ensure that investments in such solutions do not harm current and upcoming regulations.

Summarizing this…. there’s a lot in place and I was thrilled to discuss this with such a lot of smart people and companies laying the foundation to – what will be – Industry 5.0 (or even 6.0), where digital twins, smart factories, and augmented processes are a part of every factory. It’s going to be a similar leap as the employment of assembly lines, or even the steam machine.

But…. projects in the Industrial Metaverse require an innovative management, a clear vision, well-skilled technology partners, and well educated people using and deploying such a vision.

Der Beitrag The Industrial Metaverse and its practical implications erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

Yesterday, Microsoft and NVIDIA held an interesting event in Munich on “Industrial Metaverse”, of course packed with the latest graphics, crazy computing and computing power, parallelization, and lots of generative AI that can be used to simulate, train, and build digital twins for manufacturing, automotive, etc.

In Ekaterina Sirazitdinova ‘s presentation, a very nice and catchy graphic on the subject of RAG was shown (see also below, (c) NVIDIA) and I would like to briefly lose a few thoughts on this term….

We all know the problem that GenAI systems sometimes hallucinate that people suddenly have three legs and that texts are somehow generated in a strange way. This is always quite easy to recognize in pictures, but often not in texts or “data”. In particular, if artificial data has to be generated and annotated to train models (see below), such errors can naturally propagate into the later models and lead to very unattractive effects. The wonderful old rule applies here: shit-in > shit-out. This problem is now being tackled by means of RAG.

This involves comparing data generated with GenAI (= artificial data) with real information (= real data). E.g. it is checked that people actually only have two legs instead of three. Generated data that does not conform to the model is discarded. This enrichment (= augmentation) of generated data with verified information naturally requires that the latter can also be found quickly and efficiently. Classic methods from information retrieval are used here, such as Graph databases (see also https://lnkd.in/ekTt_hYb). So you try to combine the best of two worlds.

Of course, this approach stands and falls with the fact that a review is possible at all. But since we have been working in information retrieval for decades on algorithms, concepts, databases, feature graphs and ontologies that provide such information in a (more or less) structured form, this is quite promising and could also enable approaches to the verifiability of AI.

An important point here is the need to have realistic, comprehensive, correct and annotated data sets for the training of AI. So that an AI can, for example recognizes defective screws, 10,000s of images of defective screws must be available for training. However, since we often cannot break 10,000 screws, such images are generated in all conceivable variants using GenAI and then given to the image recognition system for training. The better these are, the better the “real broken screws” will be recognized later.

In my view, RAG will play a central role in the AI environment in the future!

Der Beitrag Retrieval Augmented Generation (RAG) erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

One reason for this is that companies often place more value on the solution (as long as it works) than on the architecture, long-term viability and understanding of software. What is frightening is that programmers actually get very far with it.

I firmly believe that it is much more important to understand the “why”, i.e. the concepts behind the source code, because then the “how” comes automatically. But without the “why” we might as well let Chat GPT build our software.

That’s why my lectures at the PFH Private University of Applied Sciences Göttingenare certainlynot the easiest, but they are certainly the ones that later provide sustainable, well thought-out and efficient solutions, because my students know “why”. For me, by the way, this is an important part of “German Engineering”!

Der Beitrag Copy & Paste Programming…. erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

What sounds like a “standard” announcement, however, clearly shows some of the challenges that we must consciously recognize and accept in the AI environment:

1. the cost: a single A100 costs approx. 18,000 € in retail. So that’s a whopping 6.3 billion investment, plus the computers, racks and buildings in which all this stuff has to be installed
2. the computing power: the A100 is one of the best cards for parallel computations, especially for training neural networks. 350,000 of these cards is an incredible amount of computing power, but it’s necessary to do what, for example, is required for the system. Chat GPT at all.
3. the know-how: the operation and programming of such a supercomputer is enormously complex, labor-intensive and costly.

So. Now to “us”…. If you look at these dimensions, then the question arises as to who can afford this at all? Who can manage to invest 6 billion + team + infrastructure? The public sector? Research and teaching? The middle class? Rather not.
In research (but also in many R&D departments of companies), for example, we are pleased to be able to offer a wide range of services. It’s huge when we manage to get a project proposal through for one or two A100s to research algorithms for or the behavior of neural networks.

In the end, due to these dimensions, it will probably once again come down to a handful of big players developing AI models and exploring what is technically feasible – without having to follow the rules. An AGI will emerge sooner or later (I’m guessing before the end of this year) and we in Germany or even in Europe have no way of actively helping to shape it – unless we sign up with Meta or Microsoft…

At least we then have the use of the finished models. Because the simple-looking user interfaces of Chat GPT, Copilot, Bard, etc. are only possible thanks to the incredibly high computing power mentioned above.

After all, we can continue to research, describe, observe, develop ideas, warn and hope that the big players will adopt as much of this as possible. Without research, even these big players would be nowhere near where they are today.

We can also try to anticipate the next big issues, prepare the ground and be there from the outset: one example of this is the VR/AR Industrial Coalition, in which we have been working under EU auspices for several months to set up EU-wide framework conditions, funding and projects in the field of VR/AR/metaverse and, more recently, spatial computing.

Der Beitrag Meta and the AGI erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.

The management and organization of knowledge is an absolutely critical factor for many companies. It is not just about who knows what, but above all about how knowledge is bundled, channelled, externalized, documented, substantiated, validated and optimized. What sounds trivial at first, however, is such a major challenge in practice that Chief Knowledge Officers (CKO) have even been appointed at C-level for some time now. In today’s world, knowledge is more important than ever and is the most important asset for many companies.

If knowledge is in the heads of a few people, a company should do everything it can to “secure” these people. If knowledge is stored in databases, collaboration systems or documentation systems, it should be made as searchable and, above all, findable as possible. Either way, knowledge has to be “managed” because it is constantly changing.

We often rely heavily on our large IT systems and the phrase “I’ve stored this in SharePoint” suggests that we have a structure and can easily access the knowledge in documents at any time. But when such sharepoints become huge document graveyards, even the best search engine is useless.

We therefore need to be clear about how we want to deal with knowledge. A good model for this is Nonaka and Takeuchi’s knowledge spiral, which can be used to quickly and easily analyze company processes and identify potential gaps in knowledge transfer. A distinction is made between four phases, which are run through cyclically:

– Socialization: people talk to each other, knowledge is exchanged in dialogue, people watch others, adopt knowledge and actions

– Externalization: knowledge is written down, documented, filed

– Combination: documented knowledge is refined, enriched, commonalities found, conclusions drawn

– Internalization: we read or consume knowledge from others and learn as a result

The interaction between these phases can be wonderfully superimposed on existing business processes. A CKO would now have the task of ensuring that as few knowledge gaps as possible arise during the phase transition, that each process is also illuminated from a knowledge perspective and that the origin and source of specialist knowledge is identified.

Knowledge management systems support this, but most of the most important knowledge is still in the heads of your employees.

So take good care of them and yourself!

Happy New Year to you all.

Der Beitrag Knowledge…! erschien zuerst auf Prof. Dr.-Ing. Stefan Wagenpfeil.