{"id":723,"date":"2023-08-09T06:25:34","date_gmt":"2023-08-09T06:25:34","guid":{"rendered":"https:\/\/www.stefan-wagenpfeil.de\/smart-multimedia-information-retrieval\/"},"modified":"2024-06-10T13:19:29","modified_gmt":"2024-06-10T13:19:29","slug":"smart-multimedia-information-retrieval","status":"publish","type":"page","link":"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/","title":{"rendered":"Smart Multimedia Information Retrieval"},"content":{"rendered":"\n<p>Classic information retrieval (IR) refers to the processes associated with searching for and finding information in text collections. By and large, these are very sophisticated and capable of indexing, distributing and archiving text collections, calculating metrics and implementing retrieval models (e.g. ad hoc retrieval, filtering). The &#8220;Multimedia Information Retrieval (MMIR)&#8221; area uses these techniques to enable access to any multimedia objects (image, sound, video). Multimedia features are extracted for this purpose. These are small units of information that can represent certain properties of multimedia objects. The methods of classical IR can then be applied on the basis of these characteristics.  <\/p>\n\n<p>However, multimedia collections differ in many respects from pure text collections and must therefore be treated specifically. Smart Multimedia Information Retrieval comprises MMIR concepts that are characterized by particular efficiency (quite good), effectiveness (quite fast), explainability (quite expressive and comprehensible) and scalability (quite large and much). An overview and further information can be found here:<\/p>\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-1 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<ul>\n<li><a href=\"https:\/\/www.stefan-wagenpfeil.de\/public\/GMAF\/publications\/Journal_2022.pdf\">MDPI: Smart Multimedia Information Retrieval<\/a><\/li>\n<\/ul>\n\n\n\n<p>The concepts of Smart Multimedia Information Retrieval have been implemented as part of a general framework which is available for research and teaching purposes and also forms the basis for a number of projects.<\/p>\n<\/div>\n<\/div>\n\n<p><strong>Algorithms and metrics for highly scalable multimedia processing<\/strong><\/p>\n\n<p>Multimedia objects such as images, videos or graphics are becoming more and more detailed, the resolutions are getting higher and higher and at the same time more and more multimedia objects are being recorded. So we are not only dealing with larger objects, but also with significantly more objects, whose richness of detail represents a challenge for any kind of multimedia application. As part of various research projects, we have been able to develop algorithms and metrics for such collections that enable linear rather than exponential scaling and can therefore provide extreme performance gains, especially in large projects.  <\/p>\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\"  class=\"wp-block-file__embed\" data=\"https:\/\/www.stefan-wagenpfeil.de\/wp-content\/uploads\/2023\/12\/GC_Algo_Metrics.pdf\" type=\"application\/pdf\" style=\"width:100%;height:554px\" aria-label=\"Embed of GC_Algo_Metrics.\"><\/object><a id=\"wp-block-file--media-355e2928-a0e4-4b84-9069-031aeea4948c\" href=\"https:\/\/www.stefan-wagenpfeil.de\/wp-content\/uploads\/2023\/12\/GC_Algo_Metrics.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">GC_Algo_Metrics<\/a><\/div>\n\n<p><\/p>\n\n<p><strong>The Generic Multimedia Analysis Framework (GMAF)<\/strong><\/p>\n\n<p>The Generic Multimedia Analysis Framework (GMAF) and Multimedia Feature Graphs (MMFG) are data structures and applications that unify the representation of multimedia features and standardize the analysis and extraction of multimedia features through numerous plug-ins. Graph codes are used as the central index structure, which are characterized by extremely high efficiency with large amounts of data and can therefore be used very well in rapidly growing, large multimedia collections.  <\/p>\n\n<p>Theses and \/ or projects can be realized on the basis of the GMAF. This page therefore serves as a starting point for getting to grips with the concepts and mechanisms of GMAF, MMFG and Graph Codes. It is best to start with the following publications:  <\/p>\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-2 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<ul>\n<li><a href=\"http:\/\/www.stefan-wagenpfeil.de\/public\/GMAF\/publications\/SMAP_Conference_2020.pdf\">SMAP Conf.&#8217;20: Towards AI based MMIR<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/www.stefan-wagenpfeil.de\/public\/GMAF\/publications\/BIRDS_Conference_2021.pdf\">BIRDS Conf.&#8217;21: Query Construction&#8217;<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/www.stefan-wagenpfeil.de\/public\/GMAF\/publications\/MDPI_Journal_2021.pdf\">SMAP-Jrnl.&#8217;21: MMIR with Graph Codes<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/www.stefan-wagenpfeil.de\/public\/GMAF\/publications\/ICIVR_Conference_2021.pdf\">ICIVR Conf.&#8217;21: Graph Codes Projections<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/www.stefan-wagenpfeil.de\/public\/GMAF\/publications\/MDPI_BigData.pdf\">MDPI-BigData.&#8217;21: Effective MMIR<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/www.stefan-wagenpfeil.de\/public\/GMAF\/publications\/MDPI_Semantic.pdf\">SMAP-Jrnl.&#8217;21: Towards explainability of MMIR<\/a><\/li>\n<\/ul>\n<\/div>\n<\/div>\n\n<p>GMAF Wiki and Github Repository:<\/p>\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-3 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<ul>\n<li><a href=\"https:\/\/stefan-wagenpfeil.atlassian.net\/l\/cp\/3iXBJf8F\">GMAF Confuence WIKI (click to request access)<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/github.com\/stefanwagenpfeil\/gmaf\">GMAF @ Github<\/a><\/li>\n<\/ul>\n<\/div>\n<\/div>\n\n<p>Quick start instructions<\/p>\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-4 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<ul>\n<li><a href=\"http:\/\/www.stefan-wagenpfeil.de\/public\/GMAF\/GMAF-HowtoconstructaMMFG-230323-0819.pdf\">How to construct MMFGs<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/www.stefan-wagenpfeil.de\/public\/GMAF\/GMAF-GMAF-SOAPClient-230323-0820.pdf\">GMAF SOAP Client usage<\/a><\/li>\n\n\n\n<li><a href=\"http:\/\/www.stefan-wagenpfeil.de\/public\/GMAF\/GMAF-GMAFBuildStructure-230323-0818.pdf\">GMAF Build and Dependeiency Structure<\/a><\/li>\n<\/ul>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Classic information retrieval (IR) refers to the processes associated with searching for and finding information in text collections. By and large, these are very sophisticated and capable of indexing, distributing and archiving text collections, calculating metrics and implementing retrieval models <span class=\"readmore\"><a href=\"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/\">Continue Reading<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":578,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Smart Multimedia Information Retrieval - Prof. Dr.-Ing. Stefan Wagenpfeil<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Smart Multimedia Information Retrieval - Prof. Dr.-Ing. Stefan Wagenpfeil\" \/>\n<meta property=\"og:description\" content=\"Classic information retrieval (IR) refers to the processes associated with searching for and finding information in text collections. By and large, these are very sophisticated and capable of indexing, distributing and archiving text collections, calculating metrics and implementing retrieval models Continue Reading\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/\" \/>\n<meta property=\"og:site_name\" content=\"Prof. Dr.-Ing. Stefan Wagenpfeil\" \/>\n<meta property=\"article:modified_time\" content=\"2024-06-10T13:19:29+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.stefan-wagenpfeil.de\/wp-content\/uploads\/2024\/06\/city-2022-12-15-19-35-32-utc-2-scaled.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2560\" \/>\n\t<meta property=\"og:image:height\" content=\"1714\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/\",\"url\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/\",\"name\":\"Smart Multimedia Information Retrieval - Prof. Dr.-Ing. Stefan Wagenpfeil\",\"isPartOf\":{\"@id\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.stefan-wagenpfeil.de\/wp-content\/uploads\/2024\/06\/city-2022-12-15-19-35-32-utc-2-scaled.jpg\",\"datePublished\":\"2023-08-09T06:25:34+00:00\",\"dateModified\":\"2024-06-10T13:19:29+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/#primaryimage\",\"url\":\"https:\/\/www.stefan-wagenpfeil.de\/wp-content\/uploads\/2024\/06\/city-2022-12-15-19-35-32-utc-2-scaled.jpg\",\"contentUrl\":\"https:\/\/www.stefan-wagenpfeil.de\/wp-content\/uploads\/2024\/06\/city-2022-12-15-19-35-32-utc-2-scaled.jpg\",\"width\":2560,\"height\":1714,\"caption\":\"moving car with blur light through city at night\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Smart Multimedia Information Retrieval\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/#website\",\"url\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/\",\"name\":\"Prof. Dr.-Ing. Stefan Wagenpfeil\",\"description\":\"Software Engineering | IT-Management | Gutachter f\u00fcr Informatik\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.stefan-wagenpfeil.de\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Smart Multimedia Information Retrieval - Prof. Dr.-Ing. Stefan Wagenpfeil","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/","og_locale":"en_US","og_type":"article","og_title":"Smart Multimedia Information Retrieval - Prof. Dr.-Ing. Stefan Wagenpfeil","og_description":"Classic information retrieval (IR) refers to the processes associated with searching for and finding information in text collections. By and large, these are very sophisticated and capable of indexing, distributing and archiving text collections, calculating metrics and implementing retrieval models Continue Reading","og_url":"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/","og_site_name":"Prof. Dr.-Ing. Stefan Wagenpfeil","article_modified_time":"2024-06-10T13:19:29+00:00","og_image":[{"width":2560,"height":1714,"url":"https:\/\/www.stefan-wagenpfeil.de\/wp-content\/uploads\/2024\/06\/city-2022-12-15-19-35-32-utc-2-scaled.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/","url":"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/","name":"Smart Multimedia Information Retrieval - Prof. Dr.-Ing. Stefan Wagenpfeil","isPartOf":{"@id":"https:\/\/www.stefan-wagenpfeil.de\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/#primaryimage"},"image":{"@id":"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/#primaryimage"},"thumbnailUrl":"https:\/\/www.stefan-wagenpfeil.de\/wp-content\/uploads\/2024\/06\/city-2022-12-15-19-35-32-utc-2-scaled.jpg","datePublished":"2023-08-09T06:25:34+00:00","dateModified":"2024-06-10T13:19:29+00:00","breadcrumb":{"@id":"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/#primaryimage","url":"https:\/\/www.stefan-wagenpfeil.de\/wp-content\/uploads\/2024\/06\/city-2022-12-15-19-35-32-utc-2-scaled.jpg","contentUrl":"https:\/\/www.stefan-wagenpfeil.de\/wp-content\/uploads\/2024\/06\/city-2022-12-15-19-35-32-utc-2-scaled.jpg","width":2560,"height":1714,"caption":"moving car with blur light through city at night"},{"@type":"BreadcrumbList","@id":"https:\/\/www.stefan-wagenpfeil.de\/en\/smart-multimedia-information-retrieval\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/www.stefan-wagenpfeil.de\/en\/"},{"@type":"ListItem","position":2,"name":"Smart Multimedia Information Retrieval"}]},{"@type":"WebSite","@id":"https:\/\/www.stefan-wagenpfeil.de\/en\/#website","url":"https:\/\/www.stefan-wagenpfeil.de\/en\/","name":"Prof. Dr.-Ing. Stefan Wagenpfeil","description":"Software Engineering | IT-Management | Gutachter f\u00fcr Informatik","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.stefan-wagenpfeil.de\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/www.stefan-wagenpfeil.de\/en\/wp-json\/wp\/v2\/pages\/723"}],"collection":[{"href":"https:\/\/www.stefan-wagenpfeil.de\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.stefan-wagenpfeil.de\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.stefan-wagenpfeil.de\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.stefan-wagenpfeil.de\/en\/wp-json\/wp\/v2\/comments?post=723"}],"version-history":[{"count":1,"href":"https:\/\/www.stefan-wagenpfeil.de\/en\/wp-json\/wp\/v2\/pages\/723\/revisions"}],"predecessor-version":[{"id":724,"href":"https:\/\/www.stefan-wagenpfeil.de\/en\/wp-json\/wp\/v2\/pages\/723\/revisions\/724"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.stefan-wagenpfeil.de\/en\/wp-json\/wp\/v2\/media\/578"}],"wp:attachment":[{"href":"https:\/\/www.stefan-wagenpfeil.de\/en\/wp-json\/wp\/v2\/media?parent=723"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}