This document provides an overview of the MPEG Extensible Middleware (MXM), one of ISO/IEC MPEG’s latest achievements, defining an architecture and corresponding application programming interfaces (APIs) which enable accelerated media business developments. The paper describes the vision behind MXM, its architecture, and a high level overview of the API. Additionally, example MXM applications are given.

Today's omnipresent demand for access to multimedia content via diverse devices places new challenges on efficient content delivery. While the Scalable Video Coding (SVC) extension of Advanced Video Coding (AVC) has proven to be a useful tool for the advanced delivery of video content, it has not yet found major adoption in practice. This paper introduces the concept of SVC tunneling developed in the EU FP7 ALICANTE project, which attempts to provide device-independent access to media resources at reduced network load. For SVC tunneling, video transcoding is performed at the ingress/egress points of the network, which may impact the video quality. We show that the quality impact of these transcoding steps for a transcoding chain from MPEG-2 to SVC and back to MPEG-2 accumulates to a PSNR reduction of up to 2.1 dB for transcoding at fixed target bitrates. We also discuss research challenges and open issues in SVC tunneling.

The term “trustworthy” has a very precise connotation in the European Community’s FP7 research program. For a network to be qualified as trustworthy, it needs to be secure, reliable and resilient to attacks and operational failures. Furthermore, quality of service must be guaranteed, while protecting user data, ensuring privacy and providing usable and trusted tools to support users in their security management. As such, the Trustworthy Internet not only has to include mechanisms, architectures and networking infrastructures that intrinsically provide basic security guarantees, but it also has to ensure users, service providers and application providers alike that their requirements in terms of Quality of Experience, manageability and efficiency are fully met. Providing such combined guarantees in a rapidly evolving, complex infrastructure such as the Internet requires solving challenging issues that encompass many fields of theoretical and applied information engineering. These issues span all levels of the protocol stack, ranging from finding new intrinsically secure transmission systems, to radically novel routing models, to new architectures for data dissemination and for interconnecting an unprecedented number of devices and appliances. This book aims at representing a view of the state of the “Trustworthy Internet” as we enter the second decade of our century. The material included in this book originated from the 21st International Tyrrhenian Workshop on Digital Communications, an event traditionally organized by CNIT, the Italian inter-university consortium for telecommunication research. The workshop comprised either invited contributions from renowned researchers with complementary expertise, as well as independent, peer-reviewed contributions stimulated through an open call for papers. This volume includes a selected subset of the workshop papers. Each contribution has been edited and extended after the workshop, taking into account the discussions carried out during the event, incorporating when appropriate additional technical material. Furthermore, the authors have strived to complement the specific technical aspects they present with background material devised to more comprehensively introduce the reader to the specific topic of trustworthiness tackled.

This book provides a comprehensive examination of the use of MPEG-2, MPEG-4, MPEG-7, MPEG-21, and MPEG-A standards, providing a detailed reference to their application. In this book, the authors address five leading MPEG standards, focusing not only on the standards themselves, but specifically upon their application (e.g. for broadcasting media, personalised advertising and news, multimedia collaboration, digital rights management, resource adaptation, digital home systems, and so on); including MPEG cross-breed applications. In the evolving digital multimedia landscape, this book provides comprehensive coverage of the key MPEG standards used for generation and storage, distribution and dissemination, and delivery of multimedia data to various platforms within a wide variety of application domains. It considers how these MPEG standards may be used, the context of their use, and how supporting and complementary technologies and the standards interact and add value to each other. Key Features: * Integrates the application of five popular MPEG standards (MPEG-2, MPEG-4, MPEG-7, MPEG-21, and MPEG-A) into one single volume, including MPEG cross-breed applications * Up-to-date coverage of the field based on the latest versions of the five MPEG standards * Opening chapter provides overviews of each of the five MPEG standards * Contributions from leading MPEG experts worldwide * Includes an accompanying website with supporting material (www.wiley.com/go/angelides_mpeg) This book provides an invaluable reference for researchers, practitioners, CTOs, design engineers, and developers. Postgraduate students taking MSc, MRes, MPhil and PhD courses in computer science and engineering, IT consultants, and system developers in the telecoms, broadcasting and publishing sectors will also find this book of interest.

The term Universal Multimedia Experience (UME) has gained momentum and is well recognized within the research community. As this approach puts the user into the center stage, additional complexity is added to the overall quality assessment problem which calls for a scientific framework to capture, measure, quantify, judge, and explain the user experience. In previous work we have proposed the annotation of multimedia content with sensory effect metadata that can be used to stimulate also other senses than vision or audition. In this paper we report first results obtained from subjective tests in the area of sensory effects attached to traditional multimedia presentations such as movies that shall lead to an enhanced, unique, and worthwhile user experience.

In previous and related work sensory effects are presented as a tool for increasing the user experience of multimedia presentations by stimulating also other senses than vision or audition. In this paper we primarily investigated the relationship of the Quality of Experience (QoE) due to various video bit-rates of multimedia contents annotated with sensory effects (e.g., wind, vibration, light). Therefore, we defined a subjective quality assessment methodology based on standardized methods. The paper describes the test environment, its setup, and conditions in detail. Furthermore, we experimented with a novel voting device that allows for continuous voting feedback during a sequence in addition to the overall quality voting at the end of each sequence. The results obtained from the subjective quality assessment are presented and discussed thoroughly. In anticipation of the results we can report an improvement of the quality of the multimedia experience thanks to the sensory effects.

In the past years the amount of multimedia content on the Internet or in home networks has been drastically increasing. Instead of buying traditional media (such as CDs or DVDs) users tend to buy online media. This leads to the difficulty of managing the content (e.g., movies, images). A vast amount of tools for content management exists but they are mainly focusing on one type of content (e.g., only images). Furthermore, most of the available tools are not configurable to the user’s preferences and cannot be accessed by different devices (e.g., TV, computer, mobile phone) in the home network. In this paper we present a UPnP A/V-based system for managing and sharing audio/visual content in home environments which is configurable to the user’s preferences. Furthermore, the paper depicts how this system can be used to improve the user experience by using MPEG-V.

The World Wide Web (WWW) is one of the main entry points to access and consume Internet content in various forms. In particular, the Web browser is used to access different types of media (i.e., text, image, audio, and video) and on some platforms is the only way to access the vast amount of information on the Web. Recently, it has been proposed to stimulate also other senses than vision or audition while consuming multimedia content through so- called sensory effects, with the aim to increase the user’s Quality of Experience (QoE). The effects are represented as Sensory Effects Metadata (SEM) which is associated to traditional multimedia content and is rendered (synchronized with the media) on sensory devices like fans, vibration chairs, lamps, etc. In this paper we provide a principal investigation of whether the sensory effects are ready for the WWW and, in anticipation of the result, we propose how to embed sensory effect metadata within Web content and the synchronized rendering thereof.

In this paper we describe a metadata solution for a Peer-to-Peer (P2P) content distribution system termed NextShare. We outline the key motivating factors for our approach, detail the overall generic architecture we have developed and present the workflow for delivering metadata through Peer-to-Peer based content distribution. The paper also presents the metadata model we have developed and we describe in detail how all the content can be packetized and distributed using NextShare. Finally, a description of the core and optional metadata attributes which may be utilized within the system is provided.

MPEG has developed various technologies for multimedia transport, such as MPEG-2 Transport Stream (TS) and ISO Media Base File Format. These technologies have been widely accepted and heavily used by various industries and applications, such as digital broadcasting, audio and video transport over the Internet, mobile phones, etc. In recent years, the Internet has become an important channel for the delivery of multimedia. As HTTP is widely used on the Internet, it has recently been used extensively for the delivery of multimedia content. However, there is no standard for HTTP- based streaming of MPEG media. MPEG intends to standardize a solution that addresses this need. This paper provides an overview of the recent Call of Proposals (CfP) for HTTP Streaming of MPEG Media, a new work item within ISO/IEC MPEG. In particular, it will provide an overview of existing systems and the outcome of the evaluation of the aforementioned CfP which has been conducted during the 93rd MPEG meeting in July 2010.

Editor’s Note: Media applications are becoming increasingly complex. They handle many data formats, run across multiple platforms, and support a wide range of functions. This article describes a standardized set of protocols and APIs that provides efficient access to individual system components, enables rapid deployment of new applications, and improves portability. —Anthony Vetro

In this paper, we present a multimedia test-bed enabling session mobility in the context of the emerging ISO/IEC MPEG standard, Dynamic Adaptive Streaming over HTTP (DASH). In general, session mobility is defined as the transfer of a running streaming session from one device to another device where it may need to be consumed in an adaptive way. The two main challenges are: (1) taking into account the new context of the device (e.g., capabilities) to which the session is transferred and (2) performing the actual transfer in a seamless and interoperable way. Our system addresses both challenges supported by a prototype implementation integrated into VLC. In anticipation of the results we can conclude that interoperability is achieved adopting existing standards while the performance of the system does not depend on these standards. That is, the modules responsible for the performance are usually not defined within such standards and left out for competition. However, our system is designed in an extensible way and is able to accommodate this fact.

The demand for access to advanced, distributed media resources is nowadays omnipresent due to the availability of Internet connectivity almost anywhere, anytime, and with a huge amount of different devices. This calls for rethinking of the current Internet architecture by making the network aware of which content is actually transported. This paper introduces Scalable Video Coding (SVC) as a tool for Content-Aware Networks (CANs) which is currently researched as part of the EU FP7 ALICANTE project. The architecture of ALICANTE with respect to SVC and CAN is reviewed, use cases are described, and, finally, research challenges and open issues are discussed.

The aim of the MPEG‐21 standard, the so‐called Multimedia Framework, is to enable transparent and augmented use of multimedia resources across a wide range of networks, devices, user preferences, and communities, notably for trading (of bits). As such, it provides an important step in MPEG's standards evolution, i.e., the transaction of Digital Items among Users. This paper provides an overview of applications making use specifically of MPEG‐21 Digital Items and a more in‐depth presentation of a few selected applications in research and practice.

Nowadays, mobile devices have implemented several transmission technologies which enable access to the Internet and increase the bit rate for data exchange. Despite modern mobile processors and high-resolution displays, mobile devices will never reach the stage of a powerful notebook or desktop system (for example, due to the fact of battery powered CPUs or just concerning the small-sized displays). Due to these limitations, the deliverable content for these devices should be adapted based on their capabilities including a variety of aspects (e.g., from terminal to network characteristics). These capabilities should be described in an interoperable way. In practice, however, there are many standards available and a common mapping model between these standards is not in place. Therefore, in this paper we describe such a mapping model and its implementation aspects. In particular, we focus on the whole delivery context (i.e., terminal capabilities, net- work characteristics, user preferences, etc.) and investigated the two most prominent state-of-the-art description schemes, namely User Agent Profile (UAProf) and Usage Environment Description (UED).

In this tutorial we present immersive future media technologies ranging from 3D video to sensory experiences. The former targets stereo and multi-view video technologies whereas the latter aims at stimulating other senses than vision or audition enabling an advanced user experiences through sensory effects.

This paper proposes a novel combination of arti- ficial intelligence planning and other techniques for improv- ing decision-making in the context of multi-step multime- dia content adaptation. In particular, it describes a method that allows decision-making (selecting the adaptation to perform) in situations where third-party pluggable multi- media conversion modules are involved and the multime- dia adaptation planner does not know their exact adapta- tion capabilities. In this approach, the multimedia adapta- tion planner module is only responsible for a part of the required decisions; the pluggable modules make additional decisions based on different criteria. We demonstrate that partial decision-making is not only attainable, but also in- troduces advantages with respect to a system in which these conversion modules are not capable of providing additional decisions. This means that transferring decisions from the multi-step multimedia adaptation planner to the pluggable conversion modules increases the flexibility of the adapta- tion. Moreover, by allowing conversion modules to be only partially described, the range of problems that these modules can address increases, while significantly decreasing both the description length of the adaptation capabilities and the planning decision time. Finally, we specify the conditions under which knowing the partial adaptation capabilities of a set of conversion modules will be enough to compute a proper adaptation plan.

The distribution of layered content over peer-to-peer networks becomes more important today as the users are consuming the content on terminals with various display capabilities and different network connections. For single-layer content distribution, the piece-picking algorithm only needs to ensure that content pieces are downloaded in time for display. When layered content is distributed over a peer-to-peer network, the piece-picking algorithm needs to be modified to ensure that the best possible quality is displayed while all desired pieces still have to be received before their deadline expires. In this paper, the piece-picking problem for layered content is analyzed and a number of piece-picking algorithms for layered content based on the solutions for the knapsack problem are presented. Furthermore, an evaluation of these algorithms is performed and possible applications are discussed.

This paper proposes a novel virtual Content-Aware Network (CAN) layer as a part of a full layered architecture, focused, but not limited to, on multimedia distribution with Quality of Services (QoS) assurance. The overall system is based on a flexible cooperation between providers, operators and end-users, enabling users to access the offered multimedia services in various contexts and also to become private content providers. The paper introduces the main concepts and architecture for the main virtual network layer (i.e., CAN), exposing its role and interfaces among overall system layers. This work is a part of the starting effort inside of a new European FP7 ICT research project, ALICANTE.

Future Media Internet is envisaged to provide the means to share and distribute (advanced) multimedia content and services with superior quality and striking flexibility, in a trusted and personalized way, improving citizens' quality of life, working conditions, edutainment and safety. Based on work that has taken place in projects ICT SEA and ICT OPTIMIX, and the Media Delivery Platforms Cluster of projects, we try to provide the challenges and the way ahead in the area of content adaptation.

This paper introduces a prototype test-bed for triggering sensory effects like light, wind, or vibration when presenting audiovisual resources, e.g., a video, to users. The ISO/IEC MPEG is currently standardizing the Sensory Effect Description Language (SEDL) for describing such effects. This language is briefly described in the paper and the testbed that is destined to evaluate the quality of the multimedia experience of users is presented. It consists of a video annotation tool for sensory effects, a corresponding simulation tool, and a real test system. Initial experiments and results on determining the color of light effects from the video content are reported.

Virtual worlds (often referred to as 3D3C for 3D visualization & navigation and the 3C’s of Community, Creation and Commerce) integrate existing and emerging (media) technologies (e.g. instant messaging, video, 3D, VR, AI, chat, voice, etc.) that allow for the support of existing and the development of new kinds of networked services. The emergence of virtual worlds as platforms for networked services is recognized by businesses as an important enabler as it offers the power to reshape the way companies interact with their environments (markets, customers, suppliers, creators, stakeholders, etc.) in a fashion comparable to the Internet and to allow for the development of new (breakthrough) business models, services, applications and devices. Each virtual world however has a different culture and audience making use of these specific worlds for a variety of reasons. These differences in existing Metaverses permit users to have unique experiences. In order to bridge these differences in existing and emerging Metaverses a standardized framework is required, i.e., MPEG-V Media Context and Control (ISO/IEC 23005), that will provide a lower entry level to (multiple) virtual worlds both for the provider of goods and services as well as the user. The aim of this paper is to provide an overview of MPEG-V and its intended standardization areas. Additionally, a review about MPEG-V’s most advanced part – Sensory Information – is given.

This chapter proposes a novel concept towards the deployment of a networked ‘Media Ecosystem’. The proposed solution is based on a flexible co- operation between providers, operators, and end-users, finally enabling every user first to access the offered multimedia services in various contexts, and second to share and deliver his own audiovisual content dynamically, seamlessly, and transparently to other users. Towards this goal, the proposed concept provides content-awareness to the network environment, network- and user context- awareness to the service environment, and adapted services/content to the end user for his best service experience possible, taking the role of a consumer and/or producer.

This paper describes the implementation of a VLC media player plugin enabling Dynamic Adaptive Streaming over HTTP (DASH). DASH is an emerging ISO/IEC MPEG and 3GPP standard for HTTP streaming. It aims to standardize formats enabling segmented progressive download by exploiting existing Internet infrastructure as such. Our implementation of these formats as described in this paper is based on the well-known VLC. Hence, it is fully integrated into the VLC structure and has been also submitted to the VLC development team for consideration in future releases of VLC. Therefore, it is licensed under the GNU Lesser General Public License (LGPL). The plugin provides a very flexible structure that could be easily extended with respect to different adaptation logics or profiles of the DASH standard. As a consequence, the plugin enables the integration of a variety of adaptation logics and comparison thereof, making it attractive for the research community.

MPEGs' Dynamic Adaptive Streaming over HTTP (MPEG-DASH) is an emerging standard designed for media delivery over the top of existing infrastructures and able to handle varying bandwidth conditions during a streaming session. This requirement is very important, specifically within mobile environments and, thus, DASH could potentially become a major driver for mobile multimedia streaming. Hence, this paper provides a detailed evaluation of our implementation of MPEG DASH compared to the most popular propriety systems, i.e., Microsoft Smooth Steaming, Adobe HTTP Dynamic Streaming, and Apple HTTP Live Streaming. In particular, these systems will be evaluated under restricted conditions which are due to vehicular mobility. In anticipation of the results, our prototype implementation of MPEG-DASH can very well compete with state-of-the-art solutions and, thus, can be regarded as a mature standard ready for industry adaption.

Multimedia content is increasingly used in every area of our life. Still, each type of content only stimulates the visual and/or the hearing system. Thus, the user experience depends only on those two stimuli. In this paper we introduce a standard which offers the possibility to add additional effects to multimedia content. Furthermore, we present a multimedia player and a Web browser plug-in which uses this standard to stimulate further senses by using additional sensory effects (i.e., wind, vibration, and light) to enhance the user experience resulting in a unique, worthwhile sensory experience.

This paper describes a toolset for the authoring, simulating, and rendering of multimedia content annotated with Sensory Effect Metadata (SEM) descriptions as specified in Part 3 of the MPEG V standard. This part of MPEG-V standardizes the description of sensory effects (e.g., light, wind) in order to be rendered on sensory devices (e.g., fans, vibration chairs) aiming at generating a sensory experience stimulating possibly all human senses. Our implementation comprises a toolset to author sensory effects associated with multimedia content and the simulation thereof. Furthermore, it includes a library, a standalone player, and a Web browser plug-in which enables the playback and rendering of sensory effects on off-the-shelf rendering devices and in various contexts. All software modules are available under the GNU General Public License (GPL) v3 and the GNU Lesser General Public License (LGPL) v3 respectively.

In this tutorial we present dynamic adaptive streaming over HTTP ranging from content creation to consumption. It particular, it provides an overview of the recently ratified MPEG-DASH standard, how to create content to be delivered using DASH, its consumption, and the evaluation thereof with respect to competing industry solutions. The tutorial can be roughly clustered into three parts. In part I we will provide an introduction to DASH, part II covers content creation, delivery, and consumption, and, finally, part III deals with the evaluation of existing (open source) MPEG-DASH implementations compared to state-of-art deployed industry solutions.

This paper introduces the concept of sensory experience by utilizing sensory effects such as wind or lighting as another dimension which contributes to the quality of the user experience. In particular, we utilize a representation format for sensory effects that are attached to traditional multimedia resources such as audio, video, and image contents. Sensory effects (e.g., wind, lighting, explosion, heat, cold) are rendered on special devices (e.g., fans, ambient lights, motion chair, air condition) in synchronization with the traditional multimedia resources and shall stimulate other senses than audition and vision (e.g., mechanoreception, equilibrioception, thermoreception), with the intention to increase the users Quality of Experience (QoE). In particular, the paper provides a comprehensive introduction into the concept of sensory experience, its assessment in terms of the QoE, and related standardization and implementation efforts. Finally, we will highlight open issues and research challenges including future work.

Multimedia is ubiquitously available online with large amounts of video increasingly consumed through Web sites such as YouTube or Google Video. However, online multimedia typically limits users to visual/auditory stimulus, with onscreen visual media accompanied by audio. The recent introduction of MPEG-V proposed multi-sensory user experiences in multimedia environments, such as enriching video content with so-called sensory effects like wind, vibration, light, etc. In MPEG-V, these sensory effects are represented as Sensory Effect Metadata (SEM), which is additionally associated to the multimedia content. This paper presents three user studies that utilize the sensory effects framework of MPEG-V, investigating the emotional response of users and enhancement of Quality of Experience (QoE) of Web video sequences from a range of genres with and without sensory effects. In particular, the user studies were conducted in Austria and Australia to investigate whether geography and cultural differences affect users’ elicited emotional responses and QoE.

Nowadays video is an important part of the Web and Web sites like YouTube, Hulu, etc. count millions of users consuming their content every day. However, these Web sites mainly use media players based on proprietary browser plug-ins (i.e., Adobe Flash) and do not leverage adaptive streaming systems. This paper presents a seamless integration of the recent MPEG standard on Dynamic Adaptive Streaming over HTTP (DASH) in the Web using the HTML5 video element. Therefore, we present DASHJS, a JavaScript-based MPEG-DASH client which adopts the Media Source API of Google’s Chrome browser to present a flexible and potentially browser independent DASH client. Furthermore, we present the integration of WebM based media segments in DASH giving a detailed description of the used container format structure and a corresponding Media Presentation Description (MPD). Our preliminary evaluation demonstrates the bandwidth adaption capabilities to show the effectiveness of the system.

Dynamic Adaptive Streaming over HTTP (DASH) is a convenient approach to transfer videos in an adaptive and dynamic way to the user. As a consequence, this system provides high bandwidth flexibility and is especially suitable for mobile use cases where the bandwidth variations are tremendous. In this paper we have integrated the Scalable Video Coding (SVC) extensions of the Advanced Video Coding (AVC) standard into the recently ratified MPEG-DASH standard. Furthermore, we have evaluated our solution under restricted conditions using bandwidth traces from mobile environments and compared it with an improved version of our MPEG-DASH implementation using AVC as well as major industry solutions.

Multimedia streaming technologies based on the Hypertext Transfer Protocol (HTTP) are very popular and used by many content providers such as Netflix, Hulu, and Vudu. Recently, ISO/IEC MPEG has ratified Dynamic Adaptive Streaming over HTTP (DASH) which extends the traditional HTTP streaming with an adaptive component addressing the issue of varying bandwidth conditions that users are facing in networks based on the Internet Protocol (IP). Additionally, industry has already deployed several solutions based on such an approach which simplifies large scale deployment because the whole streaming logic is located at the client. However, these features may introduce drawbacks when multiple clients compete for a network bottleneck due to the fact that the clients are not aware of the network infrastructure such as proxies or other clients. This paper identifies these negative effects and the evaluation thereof using MPEG-DASH and Microsoft Smooth Streaming. Furthermore, we propose a novel adaptation algorithm introducing the concept of fairness regarding a cluster of clients. In anticipation of the results we can conclude that we achieve more efficient bottleneck bandwidth utilization and less quality switches.

The distribution of audio-visual content over the Internet has become very popular in recent years. However, due to the bandwidth requirements of videos in high definition, a costefficient distribution is desirable. In this paper we present a low-cost consumer electronics set-top box for streaming live video content using cost-efficient P2P networks. The main features of the set-top box include content discovery, highquality playback, easy user management, and social interaction.

Adaptive HTTP streaming got lot of attention in recent years and with dynamic adaptive streaming over HTTP (DASH) a standard is available. Many papers cover this topic and present their research results, but unfortunately all of them use their own private dataset which – in most cases – is not publicly available. Hence, it is difficult to compare, e.g., adaptation algorithms in an objective way due to the lack of a common dataset which shall be used as basis for such experiments. In this paper, we present our DASH dataset featuring our DASHEncoder, an open source DASH content generation tool. We also provide basic evaluations of the different segment lengths, the influence of HTTP server settings, and, in this context, we show some of the advantages as well as problems of shorter segment lengths.

Exploring new options for publishing and content delivery offers an enormous opportunity to improve the state of the art and further modernize academic and professional publications.

In this paper we demonstrate our MPEG-DASH evaluation suite, which comprises several components on the client side as well as on the server side. The major client components are the VLC DASH plugin, libDASH, and DASH-JS, a JavaScript-based DASH client. These tools enable performance tests on various platforms, e.g., Windows and Linux as well as mobile platforms such as Android. Moreover, due to their flexible structure it is possible to integrate adaptation logics and evaluate them under consistent conditions. On the server side we provide the content generation tool DASHEncoder, our MPEG-DASH datasets well as the MPEG-DASH conformance validator.

This paper presents our peer-assisted Dynamic Adaptive Streaming over HTTP (pDASH) proposal as well as an evaluation based on our DASH simulation environment in comparison to conventional approaches, i.e., non-peer-assisted DASH. Our approach maintains the standard conformance to MPEG-DASH enabling an easy and straightforward way of enhancing a streaming system with peer assistance to reduce the bandwidth and infrastructure requirements of the content/service provider. In anticipation of the results our system achieves a bandwidth reduction of Content Distribution Networks (CDN) and as a consequence the corresponding infrastructure costs of the content/service providers by up to 25% by leveraging the upstream capacity of neighboring peers. Furthermore, the cost savings have been evaluated using a cost model that is based on the current Amazon CloudFront pricing scheme. Furthermore, we have also evaluated the performance impact that various combinations of quality levels of the content could have in a peer-assisted streaming system as well as the client behavior in such an environment.

Cloud computing is currently gaining enormous momentum due to a number of promised benefits: ease of use in terms of deployment, administration, and maintenance, along with high scalability and flexibility to create new services. However, as more personal and business applications migrate to the cloud, service quality will become an important differentiator between providers. In particular, quality of experience as perceived by users has the potential to become the guiding paradigm for managing quality in the cloud. In this article, we discuss technical challenges emerging from shifting services to the cloud, as well as how this shift impacts QoE and QoE management. Thereby, a particular focus is on multimedia cloud applications. Together with a novel QoE-based classification scheme of cloud applications, these challenges drive the research agenda on QoE management for cloud applications.

Existing and future media ecosystems need to cope with the ever-increasing heterogeneity of networks, devices, and user characteristics collectively referred to as (usage) context. The key to address this problem is media adaptation to various and dynamically changing contexts in order to provide a service quality that is regarded as satisfactory by the end user. The adaptation can be performed in many ways and at different locations, e.g., at the edge and within the network resulting in a substantial number of issues to be integrated within a media ecosystem. This paper describes research challenges, key innovations, target research outcomes, and achievements so far for edge and in-network media adaptation by introducing the concept of Scalable Video Coding (SVC) tunneling.

More and more content in various formats become available via the World Wide Web (WWW). Currently available Web browsers are able to access and interpret these contents (i.e., Web videos, text, image, and audio). These contents stimulate only senses like audition or vision. Recently, it has been proposed to stimulate also other senses while consuming multimedia content through so-called sensory effects. These sensory effects aim to enhance the ambient experience by providing effects, such as, light, wind, vibration, etc. The effects are represented as Sensory Effect Metadata (SEM) which is associated to multimedia content and is rendered on devices like fans, vibration chairs, or lamps. In this paper we present a plug-in for the Mozilla Firefox browser which is able to render such sensory effects that are provided via the WWW. Furthermore, the paper describes two user studies conducted with the plug-in and presents the results achieved.

More and more multimedia content is becoming available via the World Wide Web (WWW). These contents stimulate only senses like hearing or vision. Recently, it has been proposed to stimulate also other senses while consuming multimedia content, through so-called sensory effects. These sensory effects aim at enhancing the user’s viewing experience by providing effects such as light, wind, vibration, etc. These effects are represented as Sensory Effect Metadata (SEM) which is associated to multimedia content and is rendered on devices like fans, lamps, or vibration chairs. In this paper, we present a plug-in for Web browsers which is able to render such sensory effects provided via Web content, and we describe a demonstrator that uses this plug-in to control an amBX system.

Nowadays, mobile devices have implemented several transmission technologies which enable access to the Internet and increase the bit rate for data exchange. Despite modern mobile processors and high-resolution displays, mobile devices will never reach the stage of a powerful notebook or desktop system (for example, due to the fact of battery powered CPUs or just concerning the smallsized displays). Due to these limitations, the deliverable content for these devices should be adapted based on their capabilities including a variety of aspects (e.g., from terminal to network characteristics). These capabilities should be described in an interoperable way. In practice, however, there are many standards available and a common mapping model between these standards is not in place. Therefore, in this paper we describe such a mapping model and its implementation aspects. In particular, we focus on the whole delivery context (i.e., terminal capabilities, network characteristics, user preferences, etc.) and investigated the two most prominent state-of-the-art description schemes, namely User Agent Profile (UAProf) and Usage Environment Description (UED).

In this paper, an interoperable framework for the delivery of scalable media resources, e.g., in the standardized Scalable Video Coding (SVC) format, is presented. The framework provides support for Video on Demand (VoD) as well as multicast streaming and performs an efficient, generic, and interoperable adaptation of the streamed content based on MPEG-21 Digital Item Adaptation (DIA). The server as well as the clients of the streaming framework implement the MPEG Extensible Middleware (MXM) and utilize the MPEG Query Format (MPQF) for querying the available media resources. The framework has been fully integrated into the VLC media player. The architecture for both, VoD and multicast is presented in detail. Finally, a comparison in terms of performance of the generic MPEG-21 metadata-based adaptation approach to an SVC-specific adaptation approach is provided.

Additional constituents for the representation of multimedia content gained more and more attention. For example, the amount of cinemas equipped with additional devices (e.g., ambient light, vibrating seats, wind generators, water sprayers, heater/coolers) that stimulate senses going beyond audition and vision increases. On the content side the MPEG-V standard specifies – among others – Sensory Effect Metadata (SEM) which provides means to describe sensory effects such as wind, vibration, light, etc. to be attached to audio-visual content and, thus, offering an enhanced and immersive experience for the user. However, there is a lack of a common set of test content allowing for various subjective user studies and verification across different test sites. In this paper we provide our dataset comprising a number of videos from different genres enriched with MPEG-V compliant Sensory Effect Metadata descriptions. Furthermore, we describe possible test setups using off-the-shelf hardware for conducting subjective quality assessments.

XML-based metadata is widely adopted across the different communities and plenty of commercial and open source tools for processing and transforming are available on the market. However, all of these tools have the same requirement: they operate on plain text encoded metadata which may become a burden especially in constrained and streaming environments, e.g., when metadata needs to be processed together with multimedia content which is available in a highly efficient, binary representation format. In this paper we present techniques for transforming such kind of metadata which is encoded using the well known MPEG-7 Systems Binary Format for Metadata (BiM) without additional en-/decoding overheads, i.e., within the binary domain. As such it enables us to process both the multimedia data as well as the metadata within its compressed domain, e.g., for metadata-driven adaptation purposes within intermediary network nodes which are becoming increasingly popular in the multimedia community as well as in the XML community.

In the area of multimedia computing and communication, one of the major objectives currently being pursued is Universal Multimedia Access (UMA), i.e., enabling users to transparently access any kind of content from anywhere, anytime, with any device. In the course of the MPEG-21 (Multimedia Framework) activities, the ISO/IEC MPEG standardization group has created essential building blocks toward this goal, mainly metadata standards. Two of them are worth noting here: (1) the Digital Item Adaptation (DIA) specification, including normative vocabulary and formats to describe the multimedia consumption context (device, network, natural environment characteristics, user preferences) and to steer media adaptation operations required for UMA; (2) the Rights Expression Language (REL) and Rights Data Dictionary (RDD) specifications, providing terms and a language to express permissions on the usage of the media (who, how, what, under which conditions). In this paper, we describe a recent MPEG-21 DIA Amendment in this area, specifying description formats for: (1) multimedia conversion capabilities, which tool or service providers may use to normatively specify the media adaptation (conversion) capabilities of their tools or services, respectively; (2) permissions and conditions for multimedia conversions, which can be utilized by content providers to determine which adaptations (changes) are permitted on their contents under what conditions. The latter description format embeds media adaptation descriptions into rights expressions, filling a gap between DIA and REL/RDD. In the paper, a use case illustrating a complex UMA scenario justifies the need for these descriptions. Exemplary conversions and permissions descriptions that apply to this use case as well as detailed explanations will be given in the main portion of the paper.

In order to enable transparent and augmented use of multimedia content across a wide range of networks and devices, content adaptation is an important issue within multimedia frameworks. In this paper, we present a prototype application that receives Digital Video Broadcast (DVB) TV streams on a PC, transcodes the streams on the fly according to the individual User requirements and packs the adapted content together with available metadata into a standard compliant MPEG-21 Digital Item (DI). In this form, the framework enables the live Universal Multimedia Access (UMA) scenario where the DVB content can be transparently accessed by clients such as PCs and PDAs, anytime and anywhere.

Abstract Personalized delivery of multimedia content over the Internet opens new business perspectives for future multimedia applications and thus plays an important role in the ongoing MPEG-7 and MPEG-21 multimedia standardization efforts. Based on these standards, next-generation multimedia services will be able to automatically prepare the digital content before delivery according to the client’s device capabilities, the network conditions, or even the user’s content preferences. However, these services will have to deal with a variety of different end user devices, media formats, as well as with additional metadata when adapting the original media resources. In parallel, an increasing number of commercial or open-source media transformation tools will be available, capable of exploiting such descriptive metadata or dealing with new media formats; thus it is not realistic that a single tool will support all possible transformations. In this paper, we present a novel, fully knowledge-based approach for building such multimedia adaptation services, addressing the above mentioned issues of openness, extensibility, and concordance with existing and upcoming standards. In our approach, the original media is transformed in multiple adaptation steps performed by an extensible set of external tools, where the construction of adequate adaptation sequences is solved in an Artificial Intelligence planning process. The interoperability issue is addressed by exploiting standardized Semantic Web Services technology. This technology allows us to express tool capabilities and execution semantics in a declarative and well-defined form. In this context, existing multimedia standards serve as a shared domain ontology. The presented approach was implemented and successfully evaluated in an official ISO/IEC MPEG (Moving Picture Experts Group) Core Experiment and is currently under further evaluation by the standardization body.

The delivery and adaptation of multimedia content in dis tributed and heterogeneous environments requires flexible control and management mechanisms in terminals and in control entities inside the network. In the near future, it is important to reach interoperability between the IETF ap proaches on multimedia session establishment and control and the MPEG-21 efforts for multimedia streaming and adaptation to bring advanced multimedia service provi sioning and adaptation services towards the customer. MPEG-21 Digital Item Adaptation (DIA) provides norma tive descriptions for supporting adaptation of multimedia content, but does not define interactions with transport and control mechanisms. On the other hand, the IETF standardization efforts on multimedia session control pro vide the necessary transport (e.g. RTP) and control mechanisms (SDP/SDPng). We thus bridge the gap be tween those approaches by creating a converged XML model that enables the integration of session management and negotiation protocols (e.g. SIP or Megaco) inspired by the XML formats of MPEG-21 DIA and SDPng. We also present preliminary implementation results of the con verged model along with concepts and implementation of network-based content adaptation mechanisms through media gateways that enable flexible multimedia manage ment for heterogeneous consumer terminals.

More and more data are structured, stored, and sent over a networ using the Extensible Markup Language (XML) language. There are, however, concerns about the verbosity of XML in such a way that it may restrain further adoption of the language, especially when exchanging XML-based data over heterogeneous networks, and when it is used within constrained (mobile) devices. Therefore, alternative (binary) serialization formats of the XML data become relevant in order to reduce this overhead. However, using binary-encoded XML should not introduce interoperability issues with existing applications nor add additional complexity to new applications. On top of that, it should have a clear cost reduction over the current plain-text serialization format. A first technology is developed within the ISO/IEC Moving Picture Experts Group, namely the Binary MPEG Format for XML. It provides good compression efficiency, ability to (partially) update existingXMLtrees, and facilitates random access into, and manipulation of, the binary-encoded bit stream. Another technique is based on the Abstract Syntax Notation One specification with the Packed Encoding Rules created by the ITU-T. This paper evaluates both techniques as alternative XML serialization formats and introduces a solution for the interoperability concerns. This solution and the alternative serialization formats are validated against two real-life use cases in terms of processing speed and cost reduction. The efficiency of the alternative serialization formats are compared to a classic plain text compression technique, in particular ZIP compression.

More and more data are structured, stored, and sent over a networ using the Extensible Markup Language (XML) language. There are, however, concerns about the verbosity of XML in such a way that it may restrain further adoption of the language, especially when exchanging XML-based data over heterogeneous networks, and when it is used within constrained (mobile) devices. Therefore, alternative (binary) serialization formats of the XML data become relevant in order to reduce this overhead. However, using binary-encoded XML should not introduce interoperability issues with existing applications nor add additional complexity to new applications. On top of that, it should have a clear cost reduction over the current plain-text serialization format. A first technology is developed within the ISO/IEC Moving Picture Experts Group, namely the Binary MPEG Format for XML. It provides good compression efficiency, ability to (partially) update existingXMLtrees, and facilitates random access into, and manipulation of, the binary-encoded bit stream. Another technique is based on the Abstract Syntax Notation One specification with the Packed Encoding Rules created by the ITU-T. This paper evaluates both techniques as alternative XML serialization formats and introduces a solution for the interoperability concerns. This solution and the alternative serialization formats are validated against two real-life use cases in terms of processing speed and cost reduction. The efficiency of the alternative serialization formats are compared to a classic plain text compression technique, in particular ZIP compression.

MPEG-21 Digital Item Adaptation (DIA) has recently been finalized as part of the MPEG-21 Multimedia Framework. DIA specifies metadata for assisting the adaptation of Digital Items according to constraints on the storage, transmission and consumption, thereby enabling various types of quality of service management. This paper provides an overview of DIA, describes its use in multimedia applications, and reports on some of the ongoing activities in MPEG on extending DIA for use in rights governed environments.

Over the last decade, a wide spectrum of (multi­media) content has become available to an increasing number of users who desire to access it through various devices and over heterogeneous networks. Interoperability is the key for enabling transparent and augmented use of (multimedia) content across a wide range of networks and devices. Standardization efforts within the Moving Picture Experts Group (MPEG), in particular MPEG-7 and MPEG-21, aim to provide appropriate tools for achieving this goal of Universal Multimedia Access (UMA). This tutorial provides, in the first place, the concepts of UMA and corresponding MPEG-7 metadata tools built to support these concepts. Subsequently, the vision, an overview, and the state of the art of the emerging MPEG-21 Multimedia Framework are given. Finally, MPEG-21 Digital Item Adaptation (DIA) tools which implement the “Terminal and Networks Characteristics” key element within the whole framework are illustrated in detail. The goal of MPEG-21 DIA is to achieve interoperable transparent access to (distributed) advanced multimedia content by shielding users from network and terminal installation, configuration, management and implementation issues.

XML-based metadata is widely adopted across the different communities and plenty of commercial and open source tools for processing and transforming are available on the market. However, all of these tools have one thing in common: they operate on plain text encoded metadata which may become a burden in constrained and streaming environments, i.e., when metadata needs to be processed together with multimedia content on the fly. In this paper we present an efficient approach for transforming such kind of metadata which are encoded using MPEG's Binary Format for Metadata (BiM) without additional en-/decoding overheads, i.e., within the binary domain. Therefore, we have developed an event-based push parser for BiM encoded metadata which transforms the metadata by a limited set of processing instructions – based on traditional XML transformation techniques - operating on bit patterns instead of cost-intensive string comparisons.

Digital Item Adaptation (DIA) has been recently standardized as Part 7 of the MPEG-21 Multimedia Framework. This standard specifies tools enabling interoperable communication and adaptation of so-called Digital Items. The adaptation process becomes ever more difficult due to the heterogeneity of terminals and networks utilizing different types of multimedia contents encoded in various coding formats. Other aspects are the users� preferences and accessibility characteristics as well as the natural environment in which the content is consumed. This article describes how to use the tools within DIA in order to build a device and coding format independent adaptation module enabling interoperable multimedia communication.

The MPEG-21 standard defines a framework for the interoperable delivery and consumption of multimedia content. Within this framework the adaptation of content plays a vital role in order to support a variety of terminals and to overcome the limitations of the heterogeneous access networks. In most cases the multimedia content can be adapted by applying different adaptation operations that result in certain characteristics of the content. Therefore, an instance within the framework has to decide which adaptation operations have to be performed to achieve a satisfactory result. This process is known as adaptation decision-taking and makes extensive use of metadata describing the possible adaptation operations, the usage environment of the consumer, and constraints concerning the adaptation. Based on this metadata a mathematical optimization problem can be formulated and its solution yields the optimal parameters for the adaptation operations. However, the metadata is represented in XML resulting in a verbose and inefficient encoding. In this paper, an architecture for an Adaptation Decision-Taking Engine (ADTE) is introduced. The ADTE operates both on XML metadata and on metadata encoded with MPEG's Binary Format for Metadata (BiM) enabling an efficient metadata processing by separating the problem extraction from the actual optimization step. Furthermore, several optimization algorithms which are suitable for scalable multimedia formats are reviewed and extended where it was appropriate

XML-based metadata for digital media is becoming increasingly important, as a consequence also calling for efficient encoding and compression schemes for the storage and transport of this metadata. Moreover, support for streaming the XML metadata in conjunction with the media data is highly desirable. Such support is provided, for instance, by MPEG's Binary Format for Metadata (BiM) encoding approach, which facilitates fragmenting, delivering, and accessing the metadata in so- called Access Units (AUs). In this paper, we present a quantitative evaluation of existing XML metadata compression and encoding techniques, reaching from widely used state-of-the-art data compression algorithms to sophisticated XML-aware encoding schemes. The comparison is based on compressing MPEG-21 generic Bitstream Syntax Descriptions (gBSDs) which can grow to non-negligible sizes. The main conclusion from this investigation is that in terms of pure compression efficiency on XML files, the BiM approach (exemplified by the MPEG reference software as well as a commercial version thereof) is comparable – in terms of performance – with traditional data or specific XML compression tools. However, when XML metadata have to be fragmented, compressed, and streamed in such fragments, the results indicate that the BiM approach is superior to the other schemes.

Universal Multimedia Access (UMA) has become a driving concept behind a significant amount of research activities. One of MPEG’s (Moving Pictures Experts Group) responses to UMA is MPEG-21 Digital Item Adaptation (DIA). In this paper we present how tools as specified within DIA (i.e., normative XML-based description formats) are applied in streaming and constrained environments enabling piece-wise multimedia content adaptation including the adaptation decision- taking process and the actual resource adaptation in a coding format-independent way. Additionally, we demonstrate how the metadata overhead imposed by DIA tools can be reduced by means of appropriate metadata encoding tools.

The information revolution of the last decade has resulted in a phenomenal increase in the quantity of multimedia content available to an increasing number of different users with different preferences who access it through a plethora of devices and over heterogeneous networks. In order to address the amount of different content types, MPEG-21 Digital Item Adaptation (DIA) introduces interoperable description tools which enable coding format independent adaptation. Bandwidthefficient transport of the content to terminals with different capabilities and through a variety of access networks with various characteristics requires adaptation facilities not only on the server but also within the network. In this paper we present transport mechanisms for MPEG-21-based metadata enabling generic adaptation within the network. Three different transport mechanisms for delivering this metadata in conjunction with the corresponding multimedia content are evaluated and a payload format for the transport of this metadata is presented. Furthermore, we performed measurements which demonstrate the bandwidth benefits of our distributed adaptation approach compared to server-centric adaptation in a multicast scenario. Finally, we applied various encoding formats for the metadata which further reduces the metadata overhead.

In this paper, we present a prototype application that receives Digital Video Broadcast (DVB) TV streams on a PC, extracts semantic and syntactic metadata from the MPEG-2 Transport Stream (TS), maps them to MPEG-7 compliant descriptors, and packs these metadata and the associated multimedia content into standard compliant MPEG-21 Digital Items (DIs). In this form, i.e., as DIs, the DVB content can be searched and accessed by clients such as PCs and PDAs, and can be transported over IP networks. The DVB server can be used as a live source of multimedia content to demonstrate multimedia streaming (e.g., via RTP) and adaptation to diverse devices.

The seamless access to rich multimedia content on any device and over an network, usually known as Universal Multimedia Access, requires interoperable description tools and adaptation techniques to be developed. To address the latter issue, MPEG-21 Digital Item Adaptation (DIA) introduces the Bitstream Syntax Description (BSD) framework, which provides tools for adapting multimedia content in a generic (i.e., coding format independent) way. The basic idea is to use the eXtensible Markup Language (XML) to describe the high-level structure of a binary media bitstream, to transform its description (e.g., by means of eXtensible Stylesheet Language Transformations, XSLT), and to construct the adapted media bitstream from the transformed description. This paper presents how this basic BSD framework, initially developed for non-streamed content and suffering from inherent limitations and high memory consumption of XML-related technologies such as XSLT, can be advanced and efficiently implemented in a streaming environment and on resource-constrained devices. Two different attempts to solve the inherent problems are described. The first approach proposes an architecture based on the streamed processing of SAX (Simple Application Programming Interface for XML) events and adopts STX (Streaming Transformations for XML) as an alternative to XSLT, whereas the second approach breaks a BSD up into well-formed fragments called Process Units (PUs) that can be processed individually by a standard XSLT processor. The current status of our work as well as directions for future research are given.

In multimedia applications, XML is being increasingly used to represent metadata; examples are MPEG-7 multimedia description schemes and MPEG-21 usage environment descriptions. As with the media data, the size of, or the overhead induced by, the XML metadata is important, particularly when used on constrained mobile devices. Therefore, compression (binary encoding) of the XML data becomes relevant to reduce this overhead. Within the MPEG-7 standardization effort, a Binary Format for Metadata (BiM) was developed, ´providing good compression efficiency and facilitating random access into, and manipulation of, the binary encoded bit stream. However, using binary encoded XML should not introduce interoperability issues with existing applications, nor add additional complexity to new applications. In this paper we investigate a solution for this issue by handling the binary encoded XML data by the XML parser. As such, applications do not need to be aware of the type of encoding of the XML data. In this paper, we introduce such an XML parser and evaluate its usability in different scenarios. We measure the memory requirements and compare the processing speed of parsing binary encoded XML to plain text XML.

Digital Item Adaptation (DIA) ist der jüngste Standard im Rahmen des ISO/IEC MPEG-21 Multimedia Frameworks. Dieser Standard spezifiziert Deskriptoren und Konzepte (Tools) für die interoperable Kommunikation und ggf. Adaption von komplexen digitalen multimedialen Objekten, sog. Digital Items. Schwierigkeiten der Kommunikation und ggf. die Notwendigkeit der Adaption multimedialer Inhalte entstehen durch die Heterogenität und Beschränkungen der durchlaufenen Netzwerke (z.B. im heutigen Internet) und der benutzten Endgeräte sowie durch unterschiedliche Präferenzen und Profile der Nutzer. Dieser Artikel beschreibt eine endgeräte- und codierformat-unabhängige Komponente zur Adaption von Digital Items an verschiedene mobile Endgeräte und Netzcharakteristika.

Abstract. Universal Multimedia Access (UMA), where users can consume any multimedia resource anywhere at any time, is the driving vision of ongoing ISO/IEC Moving Picture Experts Group (MPEG) standardization efforts. In that context, intelligent adaptation means that before resources are sent over the network, they are prepared according to the client’s device capabilities, the network conditions, or even the user’s content preferences. In this paper, we argue that Semantic Web Services can serve as a key enabling technology to achieve the goals of UMA. As the standards evolve, more and more specialized software tools will be available that provide specific functionalities for adapting the media in different dimensions. When the functionality of such tools is described declaratively with the means of Semantic Web Services technology, intelligent adaptation network nodes can be developed, capable of automatically composing multi-step adaptation sequences and dynamically integrating such services available on the Web. This paper describes the architecture and a prototypical implementation of an intelligent adaptation node that supports automatic, knowledge-based service composition which is made possible by the shared domain ontology defined in MPEG metadata standards.

In order to enable transparent and augmented use of multimedia resources across a wide range of networks and devices, the forthcoming MPEG-21 standard aims at integrating the various existing technologies for delivery and consumption of digital content in a common multimedia framework. Digital Item Adaptation is one of the core concepts of the framework that will support the adaptation of multimedia resources to device capabilities, underlying network characteristics, or the user’s preferences. The scope of the standardization, however, is limited to the definition of description tools and does not deal with the internals of the adaptation process itself. In this paper, we first discuss the requirements of the resource adaptation component of an adaptation engine. These requirements include, for instance, openness for the integration of external multimedia transforming tools as well as intelligent decision taking when determining the set of required adaptation steps. We also present a prototype of a simple video resource adaptation engine that completely relies on descriptions of the resource itself (MPEG-7), the usage environment of the resource (MPEG-21), as well as declarative descriptions of the transformation tools. The prototype employs a knowledge-based engine for finding and executing the needed adaptation sequences.

In multimedia applications, XML is being increasingly used to represent metadata; examples are MPEG-7 multimedia description schemes and MPEG-21 usage environment descriptions. As with the media data, the size of, or the overhead induced by, the XML metadata is important, particularly when used on constrained mobile devices. Therefore, compression (binary encoding) of the XML data becomes relevant to reduce this overhead. Within the MPEG-7 standardization effort, a Binary Format for Metadata (BiM) was developed, providing good compression efficiency and facilitating random access into, and manipulation of, the binary encoded bit stream. In order to support processing of metadata streams in the binary domain and making this task for client applications as simple as possible, we are developing a universal parser for handling both plain text and binary encoded XML-based metadata. The parser exposes a single interface making it transparent for the application whether a plain text or a binary XML document is being processed. As part of this effort, this paper provides a detailed study of five existing XML parser models and evaluates their applicability to serve as a model for parsing binary XML data, encoded using the BiM codec. Additionally, the parser models are investigated against important usage scenarios enabled by BiM, such as dynamic updates of XML data. From the five models, two are rejected and one is only applicable for domain specific applications. Of the remaining two, one model is proposed as preferred model because of different advantages over the other model.

Due to the heterogeneity of the current terminal and network infrastructures, multimedia content needs to be adapted to specific capabilities of these terminals and network devices. Furthermore, user preferences and user environment characteristics must also be taken into consideration. The problem becomes even more complex by the diversity of multimedia content types and encoding formats. In order to meet this heterogeneity and to be applicable to different coding formats, the adaptation must be performed in a generic and interoperable way. As a response to this problem and in the context of MPEG-21, we present an approach which uses XML to describe the high-level structure of a multimedia resource in a generic way, i.e., how the multimedia content is organized, for instance in layers, frames, or scenes. For this purpose, a schema for XML-based bitstream syntax descriptions (generic Bitstream Syntax Descriptions or gBSDs) has been developed. A gBSD can describe the high-level structure of a multimedia resource in a coding format independent way. Adaptation of the resource is based on elementary transformation instructions formulated with respect to the gBSDs. These instructions have been separated from the gBSDs in order to use the same descriptions for different adaptations, e.g., temporal scaling, SNR scaling, or semantic adaptations. In the MPEG-21 framework, those adaptations can be steered for instance by the network characteristics and the user preferences. As a result, it becomes possible for coding format agnostic adaptation engines to transform media bitstreams and associated descriptions to meet the requirements imposed by the network conditions, device capabilities, and user preferences.

Access devices of all shapes and forms that can connect to different networks and be used for a myriad of applications are becoming an integral part of our everyday lives. For better or worse, these devices keep us connect at all times to our families, friends, and the office. They allow us to share our experiences and emotions, conduct business, or just say hello. With multimedia, the form of communication is much more powerful. However, we face this serious problem of heterogeneity in our terminals, in our networks and in the people that ultimately consume and interact with the information being presented to them. This chapter describes work developed within the MPEG standardization committee to help alleviate some of the burdens confronting us in connecting a wide range of multimedia content with different terminals, networks and users. Specifically, Part 7 of MPEG-21, Digital Item Adaptation (DIA) [MP21-7], is covered in this chapter.

The adaptation of multimedia resources is a common method to enable the transport and consumption of audio-visual content in constrained environments. An important aspect in this field is adaptation decision-taking, which aims to find adaptation parameters that maximize the quality for the consumer while considering the constraints of the networks and terminals involved. In this paper we focus on improving the adaptation of audio-visual content by maximizing the perceived quality. This can be realized by using a multimedia quality model and content-related metadata. We present an approach to derive this content-related metadata from subjective tests and use it for adaptation decision-taking within the MPEG-21 multimedia framework.

The streaming of multimedia content over Peer-to-Peer (P2P) networks is nowadays a well appreciated concept, as it helps distributing content to a great number of users and additionally reduces the server costs for providing the content. As the users of P2P networks often have different bandwidth connections and terminals, the same content is usually provided in different qualities. Although such a provision of the same content in different qualities helps to satisfy all users, it makes the sharing process less efficient. Users that are interested in the content in a specific quality can only exchange pieces with those users that are interested in the same content and the same quality. Thus, layered video coding, which provides different qualities within one bitstream, is especially well suited for P2P distribution. If the layered content is provided once in the best quality, all peers interested in this content can at least exchange the base layer, plus the enhancement layers they are interested in with those peers that have them available.

Scalable media contents, such as the new MPEG-4 Scalable Video Codec enable to easily retrieve different qualities of the media content by simply disregarding certain media segments. The MPEG-21-based codec-agnostic adaptation approach supports this concept by introducing an XMLbased Bitstream Syntax Description (BSD) which describes the different segments of a media content. Based on this BSD, an adaptation node can intelligently adapt any scalable media (i.e., remove specific media segments) without the need for codec-specific knowledge. The adaptation approach consists of 1) transforming this BSD and 2) adapting the media based on the transformed BSD. In this paper, we focus on the BSD transformation step and evaluate different mechanisms w.r.t. their transformation efficiency given several application scenarios. In particular, we compare the traditional stylesheet-based mechanisms with a novel mechanism based on regular expressions. We discuss both mechanisms in terms of their expressiveness, and propose how to actually employ regular expressions for codec-agnostic adaptation. Finally, we quantitatively evaluate these mechanisms in different adaptation scenarios, which vary in the size and number of required BSD units

End-to-end support for Quality of Service (QoS) has been broadly discussed in the literature. Many technologies have been proposed, each focusing on specific aspects for providing QoS guarantees to the end user. However, the integrated management of the end-to-end chain preserving QoS in heterogeneous environments is still a significant issue and insufficiently addressed to date. In this paper we propose an integrated management supervisor that takes into account the requirements from all stakeholders along the multimedia content delivery chain and provides an end-to-end management solution enabling QoS to the end user. This architecture and the subsystems that can be distributed along the end-to-end chain are detailed in this paper.

The MPEG-21 standard forms a comprehensive multimedia framework covering the entire multimedia distribution chain. In particular, it provides a flexible approach to represent, process, and transact complex multimedia objects which are referred to as Digital Items (DIs). DIs can be quite generic, independent of the application domain, and can encompass a diversity of media resources and metadata. This flexibility has an impact on the level of interoperability between systems and applications, since not all the functionality needs to be implemented. Furthermore, additional semantic rules may be implemented through the processing of the Digital Item which is possibly driven by proprietary metadata. This jeopardizes interoperability and consequently raises barriers to the successful achievement of augmented and transparent use of multimedia resources. In this context, we have investigated and evaluated the interoperability at the semantic level of Digital Items throughout the automated production, delivery and consumption of complex multimedia resources in heterogeneous environments. This paper describes the studies conducted, the experiments performed, and the conclusions reached towards that goal.

Multimedia adaptation decision-taking is referred to as the process of finding the optimal parameter settings for (multiple, possibly in series connected) multimedia content adaptation engines given the properties, characteristics, and capabilities of the content and the context in which it will be processed.

One of the future visions of multimedia networking is the provision of multimedia content at a variety of quality and price levels. Of the many approaches to this issue, one of the most predominant techniques is the concept of Perceived Quality of Service (PQoS), which extends the traditional engineering-based QoS concept to the perceptual satisfaction that the user receives from the reception of multimedia content. In this context, PQoS monitoring is becoming crucial to media service providers (SPs) for providing not only quantified PQoS-based services, but also service assurance based on multimedia content adaptation across heterogeneous networks. This work proposes a novel cross-layer monitoring architecture that utilizes a new Network QoS (NQoS) to PQoS mapping framework at the application level. The resulting QoS monitoring should allow the content delivery system to take sophisticated actions for real time media content adaptation, and aims to provide perceived service performance verification with respect to the QoS guarantees that have been specified in contractual agreements between providers and end-users. A subsequent performance evaluation of the proposed model conducted using a real test-bed environment demonstrates both the accuracy and feasibility of the network level measurements, the NQoS to PQoS mapping and the overall feasibility of the proposed end-to-end monitoring solution.

Today, there are many technologies in place to establish an infrastructure for the delivery and consumption of multimedia content. In practice, however, several elements of such an infrastructure are often stand-alone systems and a big picture of how these elements relate to each other or even fit together is not available. Therefore, MPEG-21 aims to provide an open framework for interoperable multimedia delivery and consumption. This requirement for interoperability results in a great diversity of XML-based metadata, which describes the media data on semantic or syntactic levels, in order to make it more accessible to the user. This metadata can be of considerable size, which leads to problems in streaming scenarios. Other than media data, XML metadata has no concept of samples, thus inhibiting streamed (and timed) processing, which is natural for media data. In order to address the challenges and requirements resulting from this situation, the concept of streaming instructions is introduced. These streaming instructions facilitate the fragmentation of content-related metadata, the association of media and metadata fragments with each other, and the synchronized streaming and processing of those fragments. Based on these capabilities, a dynamic and distributed multimedia content adaptation framework can be built.

Today’s increasing variety of media data results in a great diversity of XMLbased metadata, which describes the media data on semantic or syntactic levels, in order to make it more accessible to the user. This metadata can be of considerable size, which leads to problems in streaming scenarios. Other than media data, XML metadata has no concept of “samples”, thus inhibiting streamed (and timed) processing, which is natural for media data. In order to address the challenges and requirements resulting from this situation, the concept of streaming instructions is introduced. In particular, streaming instructions address the problem of fragmenting metadata, associating media segments and metadata fragments, and streaming and processing them in a synchronized manner. This is achieved by enriching the metadata with additional attributes to describe media and XML properties. Alternatively, a style sheet approach provides the opportunity to dynamically set such streaming properties without actually modifying the XML description.

A key objective of the ENTHRONE II Project is the ability to optimise the delivery of multimedia content to a wide group of heterogeneous users. One example of this is in the cooperative deployment of adaptation and caching functionality in the edge network. This hybrid approach makes it possible not only to store content locally, thus minimising the cost incurred through subsequent requests, but also to better serve heterogeneous groups of users by dynamically adapting the content to suit a wide range of terminal devices. In this paper, we describe and evaluate how the cooperative deployment of MPEG-21-based adaptation and caching of MPEG-4 SVC can result in improvements both in the quality of the content received at the user terminal and the resources consumed during the delivery

This paper describes how a multimedia adaptation framework can automatically decide the sequence of operations to be executed in order to adapt an MPEG-21 Digital Item to the MPEG-21 description of the usage environment in which it will be consumed. The main innovation of this work with respect to previous multimedia adaptation decision models is that in the proposed approach decisions can be made without knowing the exact behaviour of the operations that are going to be executed.

This paper presents a cross-layer model—formulated using interoperable description formats—for the adaptation of scalable H.264/MPEG-4 AVC (i.e., SVC) content in a video streaming system operating on aWireless LANaccess network without QoS mechanisms.SVCcontent adaptation on the server takes place on the application layer using an adaptation process compliant with the MPEG-21 Digital Item Adaptation (DIA) standard, based on input comprised of MPEG-21 DIA descriptions of content and usage environment parameters. The latter descriptions integrate information from different layers, e.g., device characteristics and packet loss rate, in an attempt to increase the interoperability of this cross-layer model, thus making it applicable to other models. For the sake of deriving model parameters, performance measurements from two wireless access point models were taken in account. Throughout the investigation it emerged that the behavior of the system strongly depends on the access point. Therefore, we investigated the use of end-to-end-based rate control algorithms for steering the content adaptation. Simulations of rate adaptation algorithms were subsequently performed, leading to the conclusion that a TFRC-based adaptation technique (TCP-Friendly Rate Control) performs quite well in adapting to limited bandwidth and varying network conditions. In the paper we demonstrate howTFRC-based content adaptation can be realized using MPEG-21 tools.

In this paper, an interoperable streaming framework for the adaptation and streaming of scalable multimedia content is presented. The streaming framework’s architecture for Video on Demand (VoD) as well as multicast streaming is presented. The VoD test-bed allows each client to set up a separate streaming session and receive the bitstreams in a quality tailored to the clients requirements, while the multicast test-bed provides a layered scalable multicast to all clients and every client needs to decide which layers to subscribe to. After the description of the architectures, a comparison in terms of performance of the MPEG-21 DIA metadata-based adaptation approach to an SVC-specific adaptation approach is presented. Furthermore, optimizations for both adaptation approaches are presented and the performance improvements due to the optimizations are discussed.

This paper presents an interoperable framework for the streaming of scalable multimedia content such as Scalable Video Coding (SVC). In particular, the framework’s architecture for both, Video on Demand (VoD) and multicast streaming, is presented. The architecture includes a detailed description of the adaptation engine – conforming to MPEG-21 Digital Item Adaptation – as well as the integration of the adaptation engine into VideoLAN’s VLC media player, which provides the streaming server and client for the framework. Following the description of the architecture, a comparison in terms of performance of the generic MPEG-21 DIA-based adaptation approach, which is utilized by the described demo, versus an SVC-specific adaptation approach is presented and possible further improvements for both approaches are investigated.

In this paper we present an interoperable multimedia delivery framework for scalable video coding based on MPEG-21 Digital Item Adaptation (DIA). In can be used to transmit scalable video contents within heterogeneous usage environments where the properties of the usage environment (e.g., terminal/network capabilities) may change dynamically during the streaming session. The usage environment is signaled by interoperable description formats provided by the DIA standard. Additionally, the adaptation itself is done by exploiting the standard's generic adaptation approach, i.e., independent of the actual coding format. Thus, the overall framework is also applicable for other scalable coding formats.

This paper provides a performance analysis of adaptation approaches designed for scalable media resources. In particular, we investigate the streaming of media resources compliant to the Scalable Video Coding (SVC) extensions of Advanced Video Coding (AVC)within heterogeneous environments, i.e., terminals and networks with different capabilities. Therefore, we have developed a test-bed in order to analyze two different approaches for the adaptation of scalable media resources, namely a generic approach that is applicable independently of the actual scalable coding format used and a specific approach especially built for SVC.The results show that if adaptation is required the generic approach clearly outperforms the approach specifically built for SVC

MPEG-21 Digital Item Adaptation (DIA) allows for a media codec agnostic multimedia adaptation approach which enables the implementation of generic adaptation engines. However, DIA is optimized for static, server-based adaptation. In this paper we introduce novel mechanisms to extend the DIA approach towards dynamic and distributed scenarios. This facilitates the placement of generic adaptation nodes which perform media codec agnostic and dynamic adaptation anywhere along the content delivery path. To validate our work we implemented such an adaptation node and evaluate its performance.

More and more content in various formats becomes available via the WorldWideWeb (WWW). Currently availableWeb browsers are able to access and interpret these contents (i.e., Web videos, text, image, and audio). These contents stimulate only senses like audition or vision. Recently, it has been proposed to stimulate also other senses while consuming multimedia content, through so-called sensory effects. These sensory effects aim to enhance the ambient experience by providing effects such as light, wind, vibration, etc. The effects are represented as Sensory Effect Metadata (SEM) description which is associated to multimedia content and is rendered on devices like fans, vibration chairs, or lamps. In this paper we present two subjective quality assessments which comprise sensory effects, such as light, in the area of the WWW and their results achieved. The first assessment evaluates the influence of light effects on the Quality of Experience (QoE). The second assessment measures the impact of different settings for the color calculation on the viewing experience. Furthermore, we describe a Web browser plug-in for Mozilla Firefox which is able to render such sensory effects that are provided via the WWW.

Streaming audio and video content currently accounts for the majority of the In⁃ternet traffic and is typically deployed over the top of the existing infrastructure. We arefacing the challenge of a plethora of media players and adaptation algorithms showing dif⁃ferent behavior but lacking a common framework for both objective and subjective evalua⁃tion of such systems. This paper aims to close this gap byproposing such a framework,de⁃scribing its architecture,providing an example evaluation, anddiscussing open issues.

Recent research efforts have reported findings on user Quality of Experience (QoE) of immersive virtual reality (VR) experiences. Truly immersive multimedia experiences also include multisensory components such as factional, tactile etc., in addition to audiovisual stimuli. In this context, this paper reports the results of a user QoE study of an olfaction-enhanced immersive VR environment. The results presented compare the user QoE between two groups (VR vs VR + Olfaction) and consider how the addition of olfaction affected user QoE levels (considering sense of enjoyment, immersion and discomfort). Self-reported measures via post-test questionnaire (10 questions) only revealed one statistically significant difference between the groups; in terms of how users felt with respect to their senses being stimulated. The presence of olfaction in the VR environment did not have a statistically significant effect in terms of user levels of enjoyment, immersion and discomfort.

In the past decade we observed the transition from push-based, fully managed media streaming to pull-based, unmanaged adaptive HTTP streaming thanks to enhancements in media compression, network capacity, and client capabilities. Adaptive media players, specifically their algorithms, have been subject to research for a long time and lead to various approaches documented in the literature. In the past years we witnessed more and more commercial deployments taking into account findings presented in scientific papers but a quantitative evaluation and assessments of its performance is missing. In this paper, we propose means for the automated performance evaluation of commercially deployed adaptive media players with respect to i) objective, well-known metrics, such as bitrate, stalls, startup delay and ii) derived/calculated metrics (instability, inefficiency, average bitrate) previously proposed in the literature. Additionally, we propose a new metric (Bandwidth index) to measure the effectiveness of bandwidth utilization and together with existing QoE models for adaptive HTTP streaming (focusing on stalls, startup delay) we demonstrate its usefulness in this domain.

Currently, we witness dramatically increasing interest in immersive media technologies like Virtual Reality (VR), particularly in omnidirectional video (OV) streaming. Omnidirectional (also called 360-degree) videos are panoramic spherical videos in which the user can look around during playback and which therefore can be understood as hybrids between traditional movie streaming and interactive VR worlds. Unfortunately, streaming this kind of content is extremely bandwidth intensive (compared to traditional 2D video) and therefore, Quality of Experience (QoE) tends to deteriorate significantly in absence of continuous optimal bandwidth conditions. In this paper, we present a first approach towards subjective QoE assessment for omnidirectional video (OV) streaming. We present the results of a lab study on the QoE impact of stalling in the context of OV streaming using head-mounted displays (HMDs). Our findings show that subjective testing for immersive media like OV is not trivial, with even simple cases like stalling leading to unexpected results. After a discussion of characteristic pitfalls and lessons learned, we provide a a set of recommendations for upcoming OV assessment studies.

Forecasts predict that Internet traffic will continue to grow in the near future. A huge share of this traffic is caused by multimedia streaming. The Quality of Experience (QoE) of such streaming services is an important aspect and in most cases the goal is to maximize the bit rate which -- in some cases -- conflicts with the requirements of both consumers and providers. For example, in mobile environments users may prefer a lower bit rate to come along with their data plan. Likewise, providers aim at minimizing bandwidth usage in order to reduce costs by transmitting less data to users while maintaining a high QoE. Today's adaptive video streaming services try to serve users with the highest bit rates which consequently results in high QoE. In practice, however, some of these high bit rate representations may not differ significantly in terms of perceived video quality compared to lower bit rate representations. In this paper, we present a novel approach to determine the statistically indifferent quality variation (SIQV) of adjacent video representations for adaptive video streaming services by adopting standard objective quality metrics and existing QoE models. In particular, whenever the quality variation between adjacent representations is imperceptible from a statistical point of view, the representation with higher bit rate can be substituted with a lower bit rate representation. As expected, this approach results in savings with respect to bandwidth consumption while still providing a high QoE for users. The approach is evaluated subjectively with a crowdsourcing study. Additionally, we highlight the benefits of our approach, by providing a case study that extrapolates possible savings for providers.

Real-time entertainment services such as streaming audio- visual content deployed over the open, unmanaged Internet account now for more than 70% during peak periods. More and more such bandwidth hungry applications and services are proposed like immersive media services such as virtual reality and, specifically omnidirectional/360-degree videos. The adaptive streaming of omnidirectional video over HTTP imposes an important challenge on today’s video delivery infrastructures which calls for dedicated, thoroughly designed techniques for content generation, delivery, and consumption. This paper describes the usage of tiles — as specified within modern video codecs such HEVC/H.265 and VP9 — enabling bandwidth efficient adaptive streaming of omnidirectional video over HTTP and we define various streaming strategies. Therefore, the parameters and characteristics of a dataset for omnidirectional video are proposed and exemplary instanti- ated to evaluate various aspects of such an ecosystem, namely bitrate overhead, bandwidth requirements, and quality as- pects in terms of viewport PSNR. The results indicate bitrate savings from 40% (in a realistic scenario with recorded head movements from real users) up to 65% (in an ideal scenario with a centered/fixed viewport) and serve as a baseline and guidelines for advanced techniques including the outline of a research roadmap for the near future.

Today, the global share and increase of Internet traffic is largely caused by multimedia delivery, mainly encompassing video, audio and image sharing on social, news, and entertainment platforms. This fact is well known to the Internet research community, which tries to counteract by increasing the content delivery efficiency. So-called Information-Centric Networks (ICN) are of considerable interest, advertised as enablers for intelligent networks, where effective delivery is to be provided as an inherent network feature. Most research proposals in this area are evaluated in simulated environments, using simulation frameworks such as OMNeT++ or ns-3. However, simulations always have shortcomings and cannot substitute measurements in physical networks. In this demonstration, we show how to readily set up an ICN-based testbed using low-budget single-board computers to conduct comprehensive emulations. We choose the scenario of pull-based adaptive video delivery as a showcase and evaluate the performance of different client-based adaptation mechanisms at the application level and different content forwarding strategies at the network level. All of the presented tools and visualization features are provided as open source contributions to the community.

A convergence of trends is shifting the focus of quality assessment from compliance with system design goals to fulfillment of user needs or expectations in different contexts.

Streaming audio and video content currently accounts for the majority of the internet traffic and is typically deployed over the top of the existing infrastructure. We are facing the challenge of a plethora of media players and adaptation algorithms showing different behavior but lack a common framework for both objective and subjective evaluation of such systems. This paper aims to close this gap by (i) proposing such a framework, (ii) describing its architecture, (iii) providing an example evaluation, (iv) and discussing open issues.

With the growing integration of telecommunication networks, Internet of Things (IoT), and 5G networks, there is a tremendous demand for multimedia services over heterogeneous networks. According to recent survey reports, mobile video traffic accounted for 60 percent of total mobile data traffic in 2016, and it will reach up to 78 percent by the end of 2021. Users’ daily lives are inundated with multimedia services, such as online video streaming (e.g., YouTube and Netflix), social networks (e.g., Facebook, Instagram, and Twitter), IoT and machine generated video (e.g, surveillance cameras), and multimedia service providers (e.g., Over-the-Top (OTT) services). Multimedia data is thus becoming the dominant traffic in the near future for both wired and wireless networks.

Omnidirectional video (ODV) streaming applica- tions are becoming increasingly popular. They enable a highly immersive experience as the user can freely choose her/his field of view within the 360-degree environment. Current deployments are fairly simple but viewport-agnostic which inevitably results in high storage/bandwidth requirements and low Quality of Experience (QoE). A promising solution is referred to as tile- based streaming which allows to have higher quality within the user’s viewport while quality outside the user’s viewport could be lower. However, empirical QoE assessment studies in this domain are still rare. Thus, this paper investigates the impact of different tile-based streaming approaches and configurations on the QoE of ODV. We present the results of a lab-based subjective evaluation in which participants evaluated 8K omnidirectional video QoE as influenced by different (i) tile-based streaming approaches (full vs. partial delivery), (ii) content types (static vs. moving camera), and (iii) tile encoding quality levels determined by different quantization parameters. Our experimental setup is character- ized by high reproducibility since relevant media delivery aspects (including the user’s head movements and dynamic tile quality adaptation) are already rendered into the respective processed video sequences. Additionally, we performed a complementary objective evaluation of the different test sequences focusing on bandwidth efficiency and objective quality metrics. The results are presented in this paper and discussed in detail which confirm that tile-based streaming of ODV improves visual quality while reducing bandwidth requirements.

The number of bandwidth-hungry applications and services is constantly growing. HTTP adaptive streaming of audiovisual content accounts for the majority of today's internet traffic. Although the internet bandwidth increases also constantly, audio-visual compression technology is inevitable and we are currently facing the challenge to be confronted with multiple video codecs. This paper provides a practical evaluation of state of the art video codecs (i. e., AV1, AVC/libx264, HEVC/libx265, VP9/Iibvpx-vp9) for large-scale HTTP adaptive streaming services. In anticipation of the results, AV I shows promising performance compared to established video codecs. Additionally, AV I is intended to be royalty free making it worthwhile to be considered for large scale HTTP adaptive streaming services.

The number of bandwidth-hungry applications and services is constantly growing. HTTP adaptive streaming of audio-visual content accounts for the majority of today's internet traffic. Although the internet bandwidth increases also constantly, audio-visual compression technology is inevitable and we are currently facing the challenge to be confronted with multiple video codecs.This paper proposes a multi-codec DASH dataset comprising AVC, HEVC, VP9, and AV1 in order to enable interoperability testing and streaming experiments for the efficient usage of these codecs under various conditions. We adopt state of the art encoding and packaging options and also provide basic quality metrics along with the DASH segments. Additionally, we briefly introduce a multi-codec DASH scheme and possible usage scenarios. Finally, we provide a preliminary evaluation of the encoding efficiency in the context of HTTP adaptive streaming services and applications.

About 300 hours of video are uploaded to YouTube every minute. The main technology to delivery YouTube content to various clients is HTTP adaptive streaming and the majority of today's internet traffic comprises streaming audio and video. In this paper, we investigate content provisioning for HTTP adaptive streaming under predefined aspects representing content features and upload characteristics as well and apply it to YouTube. Additionally, we compare the YouTube's content upload and processing functions with a commercially available video encoding service. The results reveal insights into YouTube's content upload and processing functions and the methodology can be applied to similar services. All experiments conducted within the paper allow for reproducibility thanks to the usage of open source tools, publicly available datasets, and scripts used to conduct the experiments on virtual machines.

Streaming audio and video content currently accounts for the majority of the internet traffic and is typically deployed over the top of the existing infrastructure. We are facing the challenge of a plethora of media players and adaptation algorithms showing different behavior but lack a common framework for both objective and subjective evaluation of such systems. This paper aims to close this gap by (i) proposing such a framework, (ii) describing its architecture, (iii) providing an example evaluation, (iv) and discussing open issues.

The original blog post can be found at the Bitmovin Techblog and has been modified/updated here to focus on and highlight research aspects.

In this chapter, we present Merge and Forward, an IDMS scheme for adaptive HTTP streaming as a distributed control scheme and adopting the MPEG-DASH standard as representation format. We introduce so-called IDMS sessions and describe how an unstructured peer-to-peer overlay can be created using the session information using MPEG-DASH. We objectively assess the performance of Merge and Forward with respect to convergence time (time needed until all clients hold the same reference time stamp) and scalability. After the negotiation on a reference time stamp, the clients have to synchronize their multimedia playback to the agreed reference time stamp. In order to achieve this, we propose a new adaptive media playout approach minimizing the impact of playback synchronization on the QoE. The proposed adaptive media playout is assessed subjectively using crowd sourcing. We further propose a crowd sourcing methodology for conducting subjective quality assessments in the field of IDMS by utilizing GWAP. We validate the applicability of our methodology by investigating the lower asynchronism threshold for IDMS in scenarios like online quiz games.

Research on Quality of Experience (QoE) has advanced significantly in recent years and attracts attention from various stakeholders. Different facets have been addressed by the research community like subjective user studies to identify QoE influence factors for particular applications like video streaming, QoE models to capture the effects of those influence factors on concrete applications, QoE monitoring approaches at the end user site but also within the network to assess QoE during service consumption and to provide means for QoE management for improved QoE. However, in order to progress in the area of QoE, new research directions have to be taken. The application of QoE in practice needs to consider the entire QoE eco-system and the stakeholders along the service delivery chain to the end user.

High-quality point clouds have recently gained interest as an emerging form of representing immersive 3D graphics. Unfortunately, these 3D media are bulky and severely bandwidth intensive, which makes it difficult for streaming to resource-limited and mobile devices. This has called researchers to propose efficient and adaptive approaches for streaming of high-quality point clouds.In this paper, we run a pilot study towards dynamic adaptive point cloud streaming, and extend the concept of dynamic adaptive streaming over HTTP (DASH) towards DASH-PC, a dynamic adaptive bandwidth-efficient and view-aware point cloud streaming system. DASH-PC can tackle the huge bandwidth demands of dense point cloud streaming while at the same time can semantically link to human visual acuity to maintain high visual quality when needed. In order to describe the various quality representations, we propose multiple thinning approaches to spatially sub-sample point clouds in the 3D space, and design a DASH Media Presentation Description manifest speci.c for point cloud streaming. Our initial evaluations show that we can achieve signi.cant bandwidth and performance improvement on dense point cloud streaming with minor negative quality impacts compared to the baseline scenario when no adaptations is applied.

Adaptive video streaming over HTTP is becoming omnipresent in our daily life. In the past, dozens of research papers have proposed novel approaches to address different aspects of adaptive streaming and a decent amount of player implementations (commercial and open source) are available. However, state of the art evaluations are sometimes superficial as many proposals only investigate a certain aspect of the problem or focus on a specific platform – player implementations used in actual services are rarely considered. HTML5 is now available on many platforms and foster the deployment of adaptive media streaming applications. We propose a common evaluation framework for adaptive HTML5 players and demonstrate its applicability by evaluating eight different players which are actually deployed in real-world services.

Today we can observe a plethora of adaptive video stream- ing services and media players which support interoperable formats like DASH and HLS. Most of the players and their rate adaptation algorithms work as a black box. We have de- veloped a system for easy and rapid testing of media players under various network scenarios. In this paper, we introduce AdViSE, the Adaptive Video Streaming Evaluation frame- work for the automated testing of adaptive media players. The presented framework is used for the comparison and testing of media players in the context of adaptive video streaming over HTTP in web/HTML5 environments. The demonstration showcases a series of experiments with different media players under given context conditions (e.g., network shaping, delivery format). We will also demonstrate the real-time capabilities of the framework and offline anal- ysis including several QoE metrics with respect to a newly introduced bandwidth index.

More and more immersive media applications and services are emerging on the market, but lack international standards to enable interoperability. This article provides an overview about ongoing standardization efforts in this exciting domain and highlights open research and standardization issues.

Dynamic Adaptive Streaming over HTTP (DASH) is referred to as a multimedia streaming standard to deliver high quality multimedia content over the Internet using conventional HTTP Web servers. As a fundamental feature, it enables automatic switching of quality levels according to network conditions, user requirements, and expectations. Currently, the proposed adaptation schemes for HTTP streaming mostly rely on throughput measurements and/or buffer-related metrics, such as buffer exhaustion and levels. In this paper, we propose to enhance the DASH adaptation logic by feeding it with additional information from our evaluation of the users' perception approximating the user- perceived quality of video playback. The proposed model aims at conveniently combining TCP-, buffer-, and media content-related metrics as well as user requirements and expectations to be used as an input for the DASH adaptation logic. Experiments have demonstrated that the chosen model enhances the capability of the adaptation logic to select the optimal video quality level. Finally, we integrated all our findings into a real DASH system with QoE monitoring capabilities.

Multimedia delivery over the Hypertext Transfer Protocol (HTTP) is currently very popular and with MPEGs' Dynamic Adaptive Streaming over HTTP (DASH) a standard is available to provide interoperability and enable large-scale deployments using existing infrastructures (servers, proxies, caches, etc.). This paper identifies some issue when multiple DASH clients compete for a bandwidth bottleneck when transparent proxy caches are deployed. Therefore, we propose a fair share adaptation scheme to be included within the client which – through experimental results – achieve a more efficient utilization of the bottleneck bandwidth and less quality switches.

The MPEG-DASH standard allows the client-centric access to different representations of video content via the HTTP protocol. The client can flexibly switch between different qualities, i.e., different bit rates and thus avoid waiting times during the video playback due to empty playback buffers. However, quality switches and the playback of lower qualities is perceived by the user which may reduce the Quality of Experience (QoE). Therefore, novel algorithms are required which manage the streaming behavior with respect to the user's requirements and which do not waste network resources. As indicated by recent studies, scalable video coding (SVC) may use the current network and content distribution infrastructure in a more efficient way than with single layer codecs. The contribution of this paper is the design and the implementation of a novel DASH/SVC streaming algorithm. By means of measurements in a test-bed, its performance and benefits are evaluated and compared to existing algorithms from an user-centric view point with objective performance metrics. Our findings show that the proposed algorithm outperforms other DASH mechanisms in terms of video quality, low switching frequency and usage of the available resources in a realistic mobile network scenario. This is a first step towards true QoE management of video streaming in the Internet with DASH and SVC.

Selecting and adopting the appropriate assessment method for conducting subjective quality assessments is a challenging task. The method decides whether the assessment is successful in delivering the correct answers to previously set up hypotheses. Therefore, in this paper we provide recommendations on test methods used in the domain of Sensory Experience. The proposed test methods comprise single stimulus and double stimulus methods. These test methods were used in previous studies and are presented in combination with the results of the subjective quality assessments with which they were used. Furthermore, we briefly outline our test setup, test design, and test content for assessing Sensory Experience which have been validated through conducted assessments.

Synchronizing multimedia playback among geographically distributed clients is a challenging task and is referred to as Inter-Destination Media Synchronization (IDMS). In this paper we discuss the uses cases of IDMS as identified within the SocialSensor Project and based on these use cases we derive a novel Adaptive Media Playout (AMP) scheme which aims on carrying out the process of synchronizing the media playback at the clients to a given synchronization point. We propose how visual and acoustic features can be used to achieve a QoE-aware and context-aware AMP scheme.

Preparing and conducting subjective quality assessments is a time consuming and expensive task. Therefore, we present a Web-based evaluation framework which aims on reducing the time needed for planning and designing a subjective quality assessment. The presented framework can be used for both crowdsourced and laboratory experiments. It should ease the task of designing a subjective quality assessment by providing a flexible framework. The framework has proven its applicability and flexibility to design and conduct assessments in the past and is available as open source.

Dynamic Adaptive Streaming over HTTP (DASH) is an ISO/IEC MPEG standard which enables the convenient and smooth transportation of multimedia data to heterogeneous end devices over networks with variable bandwidth conditions. This kind of streaming technology is mainly used with HTTP 1.0 and 1.1 respectively, which both have some drawbacks. Therefore, the IETF has started the development of HTTP 2.0, which is based on Google’s SPDY proposal and already supported by several major companies, e.g., Facebook, Twitter, Akamai, Mozilla and obviously Google. Furthermore, Content Centric Networking (CCN) is another novel approach for future networks that is considered as an revolutionary approach compared to HTTP 2.0. The CCN communication paradigm is completely different and does not rely on direct connections between hosts, it rather focuses on the content. This paper demonstrates DASH with HTTP 2.0/SPDY and CCN using our universal libdash library. Moreover, different mechanisms of DASH will be shown that can be used to provide on-demand and live content in an efficient and comfortable way.

MPEG Dynamic Adaptive Streaming over HTTP (DASH) is a new streaming standard that has been recently ratified as an international standard (IS). In comparison to other streaming systems, e.g., HTTP progressive download, DASH is able to handle varying bandwidth conditions providing smooth streaming. Furthermore, it enables NAT and Firewall traversal, flexible and scalable deployment as well as reduced infrastructure costs due to the reuse of existing Internet infrastructure components, e.g., proxies, caches, and Content Distribution Networks (CDN). Recently, the Hypertext Transfer Protocol Bis (httpbis) working group of the IETF has officially started the development of HTTP 2.0. Initially three major proposals have been submitted to the IETF i.e., Googles' SPDY, Microsofts' HTTP Speed+Mobility and Network-Friendly HTTP Upgrade, but SPDY has been chosen as working draft for HTTP 2.0. In this paper we implemented MPEG-DASH over HTTP 2.0 (i.e., SPDY), demonstrating its potential benefits and drawbacks. Moreover, several experimental evaluations have been performed that compare HTTP 2.0 with HTTP 1.1 and HTTP 1.0 in the context of DASH. In particular, the protocol overhead, the performance for different round trip times, and DASH with HTTP 2.0 in a lab test scenario has been evaluated in detail.

This paper provides an overview of our research conducted in the area of Sensory Experience including our implementations using MPEG-V Part 3 entitled ”Sensory Information”. MPEG-V Part 3 introduces Sensory Experience as a tool to increase the Quality of Experience by annotating traditional multimedia data with sensory effects. These sensory effects are rendered on special devices like fans, vibration chairs, ambient lights, scent disposers, water sprayers, or heating/cooling devices stimulating senses beyond the traditional ones. The paper's main focus is on the end-to-end aspects including the generation, transmission, and synchronized rendering of sensory effects with the traditional multimedia data taking movie clips as an example. Therefore, we present in this paper an open source tool chain that provides a complete end-to-end sensory effect generation and consumption framework. Furthermore, we summarize results from various subjective quality assessments conducted in this area. Finally, we point out research challenges that may encourage further research within this emerging domain.

In this paper, we present our implementation and evaluation of Dynamic Adaptive Streaming over Content centric networking (DASC) which implements MPEG Dynamic Adaptive Streaming over HTTP (DASH) utilizing a Content Centric Networking (CCN) naming scheme to identify content segments in a CCN network. In particular, video segments formatted according to MPEG-DASH are available in different quality levels but instead of HTTP, CCN is used for referencing and delivery. Based on the conditions of the network, the DASC client issues interests for segments achieving the best throughput. Due to segment caching within the network, subsequent requests for the same content can be served quicker. As a result, the quality of the video a user receives progressively improves, effectively overcoming bottlenecks in the network. We present two sets of experiments to evaluate the performance of DASC showing that throughput indeed improves. However, the generated overhead is relatively large and the adaptation strategy used for DASH that assumes an end-to-end connection could be revised for the hop-by-hop architecture of CCN.

This paper presents the usage of CCN, which is a candidate for the next-generation Internet, in combination with the new Dynamic Adaptive Streaming over HTTP (DASH) standard, which was recently ratified by ISO/IEC MPEG. In contrast to the Internet Protocol, which is mainly based on the host-to-host connection paradigm originated in the 1970s, Content Centric Networking (CCN) focuses on the content itself, instead of its location. Considering the dominance of multimedia traffic in todays' Internet, the streaming performance of DASH over CCN as well as the problems introduced by this combination is worth to be investigated in detail. Therefore, we evaluate the protocol overhead introduced by the usage of CCN compared to the HTTP versions 1.0 and 1.1. Furthermore, the performance of DASH over CCN under different network conditions is compared to the performance of HTTP 1.0/1.1. Our results showed that although CCN comes together with higher protocol overhead than HTTP 1.0/1.1 as well as a prototype implementation, it can definitely compete with HTTP 1.0 in media streaming. Based on the evaluation results, problems as well as improvement possibilities are identified, which are the basis for future work in this area.

This paper presents the usage of Content Centric Networking (CCN) for adaptive multimedia streaming in mobile environments, leveraging the recent ISO/IEC MPEG Dynamic Adaptive Streaming over HTTP (DASH) standard. The performance of DASH over CCN is evaluated using real-world mobile bandwidth traces and compared to previous evaluations of different DASH-based as well as proprietary systems. As there are no client-server connections in CCN, it offers the possibility to transfer data from multiple sources as well as over multiple links in parallel, which is definitely an important feature, e.g., for mobile devices offering multiple network links. This functionality is used and evaluated in this paper in combination with DASH, making it possible to dynamically choose the best performing link for media streaming, which is a clear advantage over DASH using HTTP and the TCP/IP protocol stack. The evaluation therefore investigates DASH over CCN in two scenarios using synthetic and real-world mobile bandwidth traces respectively, showing a significantly better performance than conventional DASH using only one connection.

The delivery of multimedia content over HTTP and on top of existing Internet infrastructures is becoming the preferred method within heterogeneous environment. The basic design principle is having an intelligent client which selects given and applicable media representations by issuing HTTP requests for individual segments based on the users' context and current conditions. Typically, this client behavior differs between implementations of the same kind and for the objective evaluations thereof appropriate datasets are needed. This paper presents a distributed dataset for the recently published MPEG-DASH standard which is mirrored at different sites across Europe, namely Klagenfurt, Paris, and Prague. A client implementation may choose to request segments from these sites and dynamically switch to a different location, e.g., in case the one currently used causes any issues. Thus, this distributed DASH dataset can be used for real-world evaluations enabling the simulation of switching between different content delivery networks.

Scalable Video Coding (SVC) in media streaming enables dynamic adaptation based on device capabilities and network conditions. In this paper, we investigate deployment options of SVC for Dynamic Adaptive Streaming over HTTP (DASH) with a special focus on scalability options, which are relevant for dynamic adaptation, especially in wireless and mobile environments. We evaluate the performance of SVC with respect to spatial and quality scalability options and compare it to non-scalable Advanced Video Coding (AVC). Performance evaluations are performed for various encoder implementations with high-definition (1080p) content. We show that a hybrid approach with multiple independent SVC bitstreams can have advantages in storage requirements at comparable rate-distortion performance.

When an adaptive media streaming system has to switch from one representation of the content to another, the switch causes viewer distraction. We introduce the concept of representation switch smoothing for alleviating the distraction and improving the overall quality of experience. As adaptive HTTP streaming systems typically deploy video buffers on the client side, the adaptation decision is known far enough ahead of playout time to perform a seamless transition between quality representations. We discuss implementation considerations for an adaptive HTTP streaming system with scalable video coding, present a subjective evaluation of the proposed approach, and identify factors that influence how smooth transitions are perceived.

Scalability within media coding allows for content adaptation towards heterogeneous user contexts and enables in-network adaptation. However, there is no straightforward solution how to encode the content in a scalable way while maximizing rate-distortion performance. In this paper we provide encoding guidelines for scalable video coding based on a survey of media streaming industry solutions and a comprehensive performance evaluation using four state of the art scalable video codecs with a focus on high-definition content (1080p).

Scalable Video Coding (SVC) in media streaming enables dynamic adaptation based on device capabilities and network conditions. In this paper, we investigate deployment options of SVC for Dynamic Adaptive Streaming over HTTP (DASH) with a special focus on scalability options, which are relevant for dynamic adaptation, especially in wireless and mobile environments. We establish encoding recommendations and evaluate the performance of SVC with respect to spatial and quality scalability options and compare it to non-scalable Advanced Video Coding (AVC). Performance evaluations are performed for various encoder implementations with high-definition (1080p) content. We show that a hybrid approach with multiple independent SVC bitstreams can have advantages in storage requirements at comparable rate-distortion performance.

Given that multimedia services are becoming increasingly popular, they are expected to play a dominant role for the Future Internet. In this context, it is essential that Content-Aware Networking (CAN) architectures, as envisaged in the frame of the Future Internet, explicitly address the efficient delivery and processing of multimedia content. This article proposes adopting a content-aware approach into the network infrastructure, thus making it capable of identifying, processing, and manipulating (i.e., adapting, caching, etc.) media streams and objects in real time towards Quality of Service/Experience (QoS/QoE) maximization. Our proposal is built upon the exploitation of scalable media coding technologies within such a content-aware networking environment and is discussed based on four representative use cases for media delivery (unicast, multicast, peer-to-peer, and adaptive HTTP streaming) and with respect to a selection of CAN challenges, specifically flow processing, caching/buffering, and QoS/QoE management.

This paper shows how adaptive streaming and on-device caching can be used to provide an always available video service. A DASH client has been extended to deal with periods of zero network connectivity, and seamlessly works in conjunction with CCN to provide local storage that is intelligently updated to provide an improved quality of experience.

Enriching multimedia with additional effects such as olfaction, light, wind, or vibration is gaining more and more momentum in both research and industry. Hence, there is the need to determine the influence of individual effects on the Quality of Experience (QoE). In this paper, we present a subjective quality assessment using the MPEG-V standard to annotate video sequences with individual sensory effects (i.e., wind, light, and vibration) and all combinations thereof. Based on the results we derive a utility model for sensory experience that accounts for the assessed sensory effects. Finally, we provide an example instantiation of the utility model and validate it against current and past results of our subjective quality assessments conducted so far.

Lately, 3D is gaining momentum in cinemas and home environments. However, 2D and 3D video content only stimulates senses like hearing and seeing. In this paper we focus on a more enhanced level of entertainment by presenting a 4D multimedia player and a corresponding demonstration setup, which stimulates further senses such as haptics using the MPEG-V: Media Context and Control standard. The presented demonstration setup uses stereoscopic 3D and sensory devices, i.e., fans, vibration panels and lights. The combination of conventional 3D content with tailored sensory effects allows us to further enhance the viewing experience of the users.

Multimedia content delivery and real-time streaming over the top of the existing infrastructure is nowadays part and parcel of every media ecosystem thanks to open standards and the adoption of the Hypertext Transfer Protocol (HTTP) as its primary mean for transportation. Hardware encoder manufacturers have adopted their product lines to support the dynamic adaptive streaming over HTTP but suffer from the inflexibility to provide scalability on demand, specifically for event-based live services that are only offered for a limited period of time. The cloud computing paradigm allows for this kind of flexibility and provide the necessary elasticity in order to easily scale with the demand required for such use case scenarios. In this paper we describe bitcodin, our transcoding and streaming-as-as-ervice platform based on open standards (i.e., MPEG-DASH) which is deployed on standard cloud and content delivery infrastructures to enable high-quality streaming to heterogeneous clients. It is currently deployed for video on demand, 24/7 live, and event-based live services using bitdash, our adaptive client framework.

Social networks have become ubiquitous and with these new possible ways for social communication and experiencing multimedia together the traditional TV scenario drifts more and more towards a distributed social experience. Asynchronism in the multimedia playback of the users may have a significant impact on the acceptability of systems providing the distributed multimedia experience. The synchronization needed in such systems is called Inter-Destination Multimedia Synchronization. In this paper we propose a demo that implements IDMS by the means of our self-organized and distributed approach assisted by pull-based streaming. We also provide a video of the planned demonstration and provide the mobile application as open source licensed under the GNU LGPL.

Modern-age technology enables us to consume multimedia for enjoyment and as a social experience. The traditional way to consume multimedia together (e.g., with family or friends in the living room) is being superseded by a location-independent scenario where geographically distributed users consume the same content while having a real-time communication channel among each other. Inter-Destination Multimedia Synchronization (IDMS) is the tool of choice in order to enable users a high-quality multimedia experience. In this paper, we investigate the influence of asynchronism when consuming multimedia content together while being geographically distributed. In particular, we adopt the concept of human computation and developed a reaction game which we used to conduct a crowdsourced subjective quality assessment in order to evaluate a threshold for multimedia synchronization within an IDMS scenario. Our results show a significant decrease in overall Quality of Experience (QOE) at an asynchronism level of 750ms. At the same time, we were able to show that asynchronism at a level of 400ms does not have significant differences regarding the QoE when compared to the synchronous reference case.

Streaming multimedia over the Internet is omnipresent but still in its infancy, specifically when it comes to the adaptation based on bandwidth/throughput measurements, clients competing for limited/shared bandwidth, and the presence of a caching infrastructure. In this paper we present a buffer-based adaptation logic in combination with a toolset of client metrics to compensate for erroneous adaptation decisions. These erroneous adaptation decisions are due to insufficient network information available at the client and issues introduced when multiple clients compete for limited/shared bandwidth and/or when caches are deployed. Our metrics enable the detection of oscillations on the client - in contrast to server-based approaches - and provide an effective compensation mechanism. We evaluate the proposed adaptation logic, which incorporates the oscillation detection and compensation method, and compare it against a throughput-based adaptation logic for scenarios comprising competing clients with and without caching enabled. In anticipation of the results, we show how the presented metrics detect oscillation periods and how such undesirable situations can be compensated while increasing the effective media throughput of the clients.

Inter-destination multimedia synchronization and quality of experience are critical to the success of social TV, which integrates television viewing with social networking.

In this tutorial we present state of the art and challenges ahead in over-the-top content delivery. It particular, the goal of this tutorial is to provide an overview of adaptive media delivery, specifically in the context of HTTP adaptive streaming (HAS) including the recently ratified MPEG-DASH standard. The main focus of the tutorial will be on the common problems in HAS deployments such as client design, QoE optimization, multi-screen and hybrid delivery scenarios, and synchronization issues. For each problem, we will examine proposed solutions along with their pros and cons. In the last part of the tutorial, we will look into the open issues and review the work-in-progress and future research directions.

This chapter introduces the concept of Sensory Experience which aims to define the Quality of Experience (QoE) going beyond audio-visual content. In particular, we show how to utilize sensory effects such as ambient light, scent, wind, or vibration as additional dimensions contributing to the quality of the user experience. Therefore, we utilize a standardized representation format for sensory effects that are attached to traditional multimedia resources such as audio, video, and image contents. Sensory effects are rendered on special devices (e.g., fans, lights, motion chair, scent emitter) in synchronization with the traditional multimedia resources and shall stimulate also other senses than hearing and seeing with the intention to increase the Quality of Experience (QoE), in this context referred to as Sensory Experience.

The emerging MPEG standard Dynamic Adaptive Streaming over HTTP (MPEG-DASH) is designed for media delivery over the top of existing infrastructures and enables smooth multimedia streaming towards heterogeneous devices including both wired and wireless environments. The MPEG-DASH standard was designed to work with HTTP-URLs but mandates neither the actual version nor which underlying protocols to be used. This paper will provide a detailed introduction into emerging protocols (HTTP/2.0 and beyond) to be used in the context of adaptive media streaming, specifically DASH.

Synchronizing multimedia playback among geographically distributed clients is a challenging task and is referred to as Inter-Destination Media Synchronization (IDMS). In this paper we discuss the uses cases of IDMS as identified within the SocialSensor Project and based on these use cases we derive a novel Adaptive Media Playout (AMP) scheme which aims on carrying out the process of synchronizing the media playback at the clients to a given synchronization point. We propose how visual and acoustic features can be used to achieve a QoE-aware and context-aware AMP scheme.

Mobile peer-to-peer (P2P) computing with applications such as for video on demand, file sharing, and video conferencing is gaining momentum based on new standards and technologies such as IETF PPSP, WiFi-Direct and BitTorrent live streaming. In this paper, we describe the mobile system MyMedia, that allows users to search, share and experience videos and live recordings using P2P and at the best quality possible with respect to available network capacity. In particular, the MyMedia system features a high-precision semantic P2P search and dynamic network-adaptive P2P live streaming of MPEG videos over HTTP based on the ISO/IEC standard MPEG-DASH from mobile to mobile devices in unstructured wireless P2P networks. These features have been integrated in the mobile application TIFF EventLive of the 54th Thessaloniki international film festival. The evaluation of their performance and device energy consumption, and a first user evaluation at the festival showed that the MyMedia system is suitable and accepted by users for its purpose in practice. The MyMedia system is available as open-source software for the Android operating system.

With the introduction of social networks like Facebook, Google+, and Twitter, the ways of sharing impressions of events has changed. We try to go a step further than social networks do. We offer the possibility to share events live with friends and colleagues. Our approach is based on semantic search in unstructured peer-to-peer (P2P) networks for querying content in mobile ad hoc networks and dynamic adaptive streaming over HTTP for the actual delivery of the real-time media impressions.

Social networks have become pervasive and have changed the way of social interaction. The traditional TV experience drifts from an event tied to a certain place with the family or friends to a location-independent and distributed social experience. Additionally, more and more video on-demand services adopt a pull-based streaming approach. In order to provide a synchronized and distributed TV experience we introduce a self-organized Inter-Destination Multimedia Synchronization (IDMS) framework for adaptive media streaming. In particular, we extend the principles of IDMS to adaptive media streaming over HTTP (i.e., MPEG-DASH) and enable a synchronized multimedia playback among geographically distributed clients. Therefore, we introduce session management to MPEG-DASH and for negotiating on a reference playback timestamp among the participating peers in an IDMS session we propose a distributed control scheme. We evaluate our proposed scheme with respect to scalability and time required for negotiating on the reference playback timestamp. Furthermore, we investigate how to compensate the identified asynchronism by using adaptive media playout with respect to the Quality of Experience (QoE). Therefore, we define a temporal distortion measure for audio and video which allows us to model the impact of playback rate variations on the QoE. This measure is evaluated by conducting a subjective quality assessment using crowdsourcing.

In the past decade Adaptive Media Playout (AMP) has been intensively studied with respect to the detection of when to increase or decrease the playback rate in order to maintain a certain buffer fill state. In this paper we subjectively assess the QoE of AMP with respect to non-periodically and randomly selected content sections of a video sequence by us- ing crowdsourcing. Furthermore, we introduce features that allow to quantify the distortion for audio and video that are caused by increasing or decreasing the playback rate. With these preliminaries we study the correlation between the introduced features and the subjectively assessed QoE. Therefore, we derive a utility model that allows to estimate the QoE with the introduced features. We instantiate and validate the model by the use of the data gathered by the conducted study.

Crowdsourced quality-of-experience (QoE) assessments are more cost-effective and flexible than traditional in-lab evaluations but require careful test design, innovative incentive mechanisms, and technical expertise to address various implementation challenges.

The popularity of the crowdsourcing for performing various tasks online increased significantly in the past few years. The low cost and flexibility of crowdsourcing, in particular, attracted researchers in the field of subjective multimedia evalua- tions and Quality of Experience (QoE). Since online assessment of multimedia content is challenging, several dedicated frameworks were created to aid in the designing of the tests, including the support of the testing methodologies like ACR, DCR, and PC, setting up the tasks, training sessions, screening of the subjects, and storage of the resulted data. In this paper, we focus on the web-based frameworks for multimedia quality assessments that support commonly used crowdsourcing platforms such as Amazon Mechanical Turk and Microworkers. We provide a detailed overview of the crowdsourcing frameworks and evaluate them to aid researchers in the field of QoE assessment in the selection of frameworks and crowdsourcing platforms that are adequate for their experiments.

ICN has received a lot of attention in recent years, and is a promising approach for the Future Internet design. As multimedia is the dominating traffic in today's and (most likely) the Future Internet, it is important to consider this type of data transmission in the context of ICN. In particular, the adaptive streaming of multimedia content is a promising approach for usage within ICN, as the client has full control over the streaming session and has the possibility to adapt the multimedia stream to its context (e.g. network conditions, device capabilities), which is compatible with the paradigms adopted by ICN. In this article we investigate the implementation of adaptive multimedia streaming within networks adopting the ICN approach. In particular, we present our approach based on the recently ratified ISO/IEC MPEG standard Dynamic Adaptive Streaming over HTTP and the ICN representative Content-Centric Networking, including baseline evaluations and open research challenges.

Mulsemedia—multiple sensorial media—captures a wide variety of research efforts and applications. This article presents a historic perspective on mulsemedia work and reviews current developments in the area. These take place across the traditional multimedia spectrum—from virtual reality applications to computer games—as well as efforts in the arts, gastronomy, and therapy, to mention a few. We also describe standardization efforts, via the MPEG-V standard, and identify future developments and exciting challenges the community needs to overcome.

In this paper, a generic method is described to allow the adaptation of different multimedia resources by a single, media resource-agnostic processor. This method is based on an XML description of the media resource’s bitstream syntax, which can be transformed to reflect the desired adaptation and then be used to generate an adapted version of the bitstream. Based on this concept, two complementary technologies, BSDL and gBS Schema, are presented. The two technologies provide solutions for parsing a bitstream to generate its XML description, for the generic structuring of this description, and the generation of an adapted bitstream using its transformed description. The two technologies can be used as stand-alone tools; however, a joint approach has been developed in order to harmonise the two solutions and exploit their strengths. Since BSDL has been presented in previous publications, this paper is focusing more on the gBS Schema and the joint BSDL/gBS Schema approach.