multimedia communications technical committee ieee...

IEEE COMSOC MMTC R-Letter

http://committees.comsoc.org/mmc 1/9 Vol. 7, No. 1, February 2016

MULTIMEDIA COMMUNICATIONS TECHNICAL COMMITTEE IEEE COMMUNICATIONS SOCIETY http://committees.comsoc.org/mmc

R-LETTER

Vol. 7, No. 1, February 2016

TABLE OF CONTENTS

Message from the Review Board Directors .................................................................... 2A General Multi-Modal Learning Framework for RGB-D Object Recognition ........ 3

A short review for “Large-Margin Multi-Modal Deep Learning for RGB-D Object Recognition” (Edited by Carl James Debono) ................................................................ 3

Energy-efficient multimedia transmission in wireless heterogeneous networks ......... 5A short review for “Energy-efficient multimedia transmissions through base station cooperation over heterogeneous cellular networks exploiting user behavior” (Edited by Yan Zhang) ..................................................................................................................... 5

Energy-Awareness and Bandwidth Efficiency in Mobile Video Streaming ................ 6A short review for “Energy-Aware and Bandwidth-Efficient Hybrid Video Streaming Over Mobile Networks” (Edited by Christian Timmerer) .............................................. 6

Paper Nomination Policy .................................................................................................. 8MMTC R-Letter Editorial Board .................................................................................... 9Multimedia Communications Technical Committee Officers ...................................... 9



Message from the Review Board Directors Welcome to the February 2016 issue of the Re-view Letter (R-Letter) of the IEEE Communica-tions Society Multimedia Communications Technical Committee (MMTC). This is the first issue of 2016 and on behalf of the review board we wish you all the best for 2016.

This issue comprises three R-Letters in the area of multimodal learning and mobile video stream-ing.

We hope that this issue stimulates your re-search in the area of multimedia communica-tion.

An overview of all R-Letters is provided in the following:

The first paper, published in IEEE Transactions on Multimedia and edited by Carl James Debone, explores a multimodal deep learning framework for RGB-D object recognition.

The second paper, published in the IEEE Wire-less Communications and edited by Yan Zhang, discusses energy-efficient multimedia transmis-sion in wireless heterogeneous networks.

The third paper is edited by Christian Timmerer and has been published within the IEEE Trans-actions on Multimedia. It also addresses the en-ergy-awareness within mobile video streaming but also includes means to reduce network load

by proposing a hybrid multicast-unicast stream-ing system.

We would like to thank all the authors, nomina-tors, reviewers, editors, and others who contrib-ute to the release of this issue.

Finally, we would like to highlight upcoming conferences in 2016 which are related to MMTC:

§ ACM Multimedia Systems, May 10-13, Klagenfurt am Wörthersee, Austria: http://mmsys2016.itec.aau.at/

§ IEEE ICC, May 23-27, Kuala Lumpur, Malaysia: http://icc2016.ieee-icc.org/

§ IEEE ICME, July 11-15, Seattle, USA: http://www.icme2016.org/

§ ACM Multimedia, October 15-19, Am-sterdam, The Netherlands: http://www.acmmm.org/2016/

§ IEEE GLOBECOM, December 4-8, Washington, DC, USA: http://globecom2016.ieee-globecom.org/

IEEE ComSoc MMTC R-Letter

Director: Christian Timmerer Alpen-Adria-Universität Klagenfurt, Austria Email: [email protected]

Co-Director: Yan Zhang Simula Research Laboratory, Norway Email: [email protected]



A General Multi-Modal Learning Framework for RGB-D Object Recognition

A short review for “Large-Margin Multi-Modal Deep Learning for RGB-D Object Recognition” (Edited by Carl James Debono)

Depth sensors are finding their way in consumer cameras. The additional data provided by the depth sensor can be exploited in object recognition systems as it captures the same scene found in the color im-age. The depth data is a greyscale image characterized by homogenous regions and sharp edges. The sharp edges represent a change in depth, typically found around objects in natural images and video. There-fore, the edges found in the depth image correspond to edges that are also present in the color image. This correlation helps in identifying objects as the depth images are not effected by light and object color vari-ations. The use of color and depth for object recognition has attracted some attention in the last few years. Looking from the feature generation point-of-view, the meth-ods found in literature can be divided into two catego-ries, namely: techniques that use hand-crafted fea-tures, such as [1] and [2], and others that use learned features, such as [3] and [4]. Algorithms like scale-invariant feature transform (SIFT) [5] and speeded up robust features (SURF) [6] are applied in hand-crafted features solutions to describe color and texture data found in color images and 3D geometry from the depth images. These are generally tuned manually for a particular application, making them dataset depend-ent. Furthermore, they capture only a subset of the cues that can be applied in recognition systems. Con-volutional-recursive deep learning [7] and convolu-tional k-means descriptors [3], amongst others, are examples of algorithms used in learned features tech-niques. These generally learn the features of the color and the depth independently or treat them in the same way. This implies that the complementary relations between the two formats are not well exploited. The authors of the original paper tackle these limita-tions by using a convolutional neural network (CNN) based multi-modal learning scheme. Deep CNNs are developed to independently learn the feature repre-sentations for the color and the depth images. These parallel streams are then connected in a final multi-modal layer. The final layer finds the most distinctive

features in the color and in the depth and connects the complementary relationship between the two repre-sentations of the scene. In particular, uncorrelated noise present only in one representation can be ig-nored and the learned features will exclude the di-mensions that are particularly susceptible to the iden-tified noise. The results from the output stage are backpropagated through the network to update the CNNs’ parameters and the process iterates until con-vergence is achieved. The authors of the original paper evaluate their solu-tion by testing its performance on the red, green, blue – depth (RGB-D) object dataset [8] and the 2D3D dataset [2]. Both a high and a low resolution version of the images were employed in the experiments. This was done by respectively rescaling all the images to 150 × 150 and 80 × 80 pixels. A slight modification in the CNN architecture is employed in the two ver-sions, with the low-resolution version having one less convolutional layer and pooling layer compared to the high-resolution one. The authors set the data batch size to 128 and the weights of the CNN convolutional and fully-connected layers were initialized with a zero-mean Gaussian distribution having a standard deviation of 0.01. The other parameters were set em-pirically and kept constant during the tests. The re-sults observed by the original authors indicate good performance when compared with other techniques found in literature. The number of features needed is also smaller than other methods, reducing the number of computations necessary. The availability of cheap cameras and depth sensors provides further data that can be exploited in object recognition within natural images and many other applications. The common features between the two image modalities help to reduce the noise in the im-age generated during its capture, either due to errors introduced by the electronic devices or naturally due to differences in colors and shadows. Furthermore, the application of CNN and other deep learning tech-niques provide tools that can improve the classifica-tion of the data. The mixed architecture, having a

A. Wang, J. Lu, J. Cai, T.-J. Cham and G. Wang, “Large-Margin Multi-Modal Deep Learning for RGB-D Object Recognition,” IEEE Transactions on Multimedia, vol. 17, no. 11, pp. 1887-1898, November 2015.



separate first stage for each image format and then combining the outputs in a common second stage, allows for better discrimination of features providing good results. Research work is still necessary to improve further the accuracy of these algorithms through the use of better features. Unsupervised solutions are also pre-ferred to further generalize the algorithms and make them more independent of the training sets. Further-more, complexity reduction is also important, and therefore solutions that demand less features and al-low for more parallel architectures need to be sought. References: [1] L. Bo, X. Ren, and D. Fox, “Depth kernel de-

scriptors for object recognition,” in Proc. IEEE/RSJ International Conference Intelligent Robots and Systems, pp. 821–826, September 2011.

[2] B. Browatzki, J. Fischer, B. Graf, H. Bulthoff, and C. Wallraven, “Going into depth: Evaluating 2D and 3D cues for object classification on a new, large-scale object dataset,” in Proc. IEEE International Conference on Computer Vision Workshops, pp. 1189–1195, November 2011.

[3] M. Blum, J. T. Springenberg, J. Wulfing, and M. Riedmiller, “A learned feature descriptor for ob-ject recognition in RGB-D data,” in Proc. IEEE International Conference on Robotics and Auto-mation, pp. 1298–1303, May 2012.

[4] K. Yu, Y. Lin, and J. Lafferty, “Learning image representations from the pixel level via hierar-chical sparse coding,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 1713–1720, Jun. 2011.

[5] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, November 2004.

[6] H. Bay, T. Tuytelaars, and L. Van Gool, “SURF: Speeded up robust features,” in Proc. of the Eu-ropean Conference Computer Vision, pp. 404–

417, 2006. [7] R. Socher, B. Huval, B. Bhat, C. D. Manning,

and A. Y. Ng, “Convolutional-recursive deep learning for 3D object classification,” in Proc. Advances in Neural Information Processing Sys-tems 25, pp. 665–673, 2012.

[8] K. Lai, L. Bo, X. Ren, and D. Fox, “A large-scale hierarchical multiview RGB-D object dataset,” in Proc. of the IEEE International Conference on Robotics and Automation, pp. 1817–1824, May 2011.

Carl James Debono (S’97, M’01, SM’07) received his B.Eng. (Hons.) degree in Electrical Engi-neering from the University of Malta, Malta, in 1997 and the Ph.D. degree in Electronics and Computer Engineering from the University of Pavia, Italy, in 2000. Between 1997 and 2001 he was

employed as a Research Engineer in the area of Inte-grated Circuit Design with the Department of Microe-lectronics at the University of Malta. In 2000 he was also engaged as a Research Associate with Texas A&M University, Texas, USA. In 2001 he was ap-pointed Lecturer with the Department of Communica-tions and Computer Engineering at the University of Malta and is now an Associate Professor. He is cur-rently the Head of the Department of Communica-tions and Computer Engineering at the University of Malta. Prof. Debono is a senior member of the IEEE and served as chair of the IEEE Malta Section between 2007 and 2010. He was the IEEE Region 8 Vice-Chair of Technical Activities between 2013 and 2014. He has served on various technical program commit-tees of international conferences and as a reviewer in journals and conferences. His research interests are in wireless systems design and applications, multi-view video coding, resilient multimedia transmission, and modeling of communication systems.



Energy-efficient multimedia transmission in wireless heterogeneous networks

A short review for “Energy-efficient multimedia transmissions through base station cooperation over heterogeneous cellular networks exploiting user behavior”

(Edited by Yan Zhang)

Mobile traffic has increased rapidly due to high-speed wireless networks, rich multimedia services and pow-erful new devices. It is estimated that there will be more than 8.6 billion handheld or personal mobile-ready devices and 1.7 billion machine-to-machine connections by 2017 [1]. Mobile multimedia, espe-cially video on demand (VoD) [2], will generate much of the mobile traffic growth through 2017. Ac-cording to Cisco’s prediction [1], mobile video will grow at a compound annual growth rate (CAGR) of 75 percent between 2012 and 2017. Meanwhile, energy saving has introduced considera-ble attention in the design of next generation wireless networks. Thus energy-efficient multimedia transmis-sion has been very important and there is an urgent need for a new disruptive paradigm of green media to bridge the gap between wireless technologies and multimedia applications. In current literatures there have been many research topics for multimedia transmission in wireless net-works. In this paper, the characteristics of both mul-timedia traffic and cellular networks are utilized in order to reduce the energy consumption due to grow-ing multimedia applications in future heterogeneous cellular networks (HCNs). The authors taken the broadcast nature of wireless transmission in cellular networks into account and argue that one of the fun-damental ideas to improve energy efficiency for mul-timedia transmission is to use broadcast/multicast to serve users with identical or similar requests. This paper proposes an energy-efficient multimedia multicast transmission scheme through macro/small BS cooperation in HCN through considering both the characteristics of wide coverage of MBSs and high data rate of SBSs, In the proposed scheme, through analyzing user behavior and collecting multimedia requests in a hotspot area, the MBS first multicasts the multimedia stream, the subsequent arriving users during a certain time window period requesting the same stream immediately join the multicast group, and the missing fraction of the stream is delivered by

the SBS through initiating separate patching streams (each missing stream is unicast for each user), since the multimedia stream is buffered at the SBS. During multimedia delivery, the macrocell and small cell cooperatively multicast/unicast the multimedia streams; hence, we call this BS cooperation. This kind of transmission is different from device-to-device (D2D) transmission mode, where the multimedia is buffered at the user side. Since both a macrocell and small cells participate in the transmission process, the interface Xn is enhanced to support real-time stream-ing transmission. In current deployment, Xn is always a fiber link, and its delay and transmission rate can be guaranteed. In analysis, a detailed description of the proposed paradigm is presented, and the optimal time window and bandwidth allocation scheme for energy saving are also derived. Analytical and simulation results show that the proposed scheme achieves good performance. Overall, the concept of user behavior proposed in this paper to describe and evaluate characteristics in het-erogeneous cellular networks is very interesting and beneficial in this area. This paper provides a possible architecture for multimedia transmission over HCNs.. And based on the architecture, an energy-efficient multimedia transmission scheme through base station cooperation is proposed to optimize the energy effi-ciency by exploiting user behavior, and heterogene-ous networks, where macrocells and small cells coop-eratively multicast/unicast the multimedia steams without QoS degradation. Through analysis and simu-lations, this paper presents results showing that the exploitation of user behavior and base station cooper-ation is able to bring green multimedia transmission to HCNs. References: [1] Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2012–2017. [2] J. Choi, A. Reaz, and B. Mukherjee, “A Survey of User Behavior in VoD Service and Bandwidth-Saving Multicast Streaming Schemes,” IEEE Commun. Sur-veys Tutorials, vol. 14, no. 1, First Quarter 2012, pp. 156–69.

Xing Zhang; Rong Yu; Yan Zhang; Yue Gao; Im, M.; Cuthbert, L.; Wenbo Wang, "Energy-efficient multimedia transmissions through base station cooperation over heterogeneous cellular networks exploiting user behavior," in Wireless Communications, IEEE , vol.21, no.4, pp.54-61, August 2014



Energy-Awareness and Bandwidth Efficiency in Mobile Video Streaming

A short review for “Energy-Aware and Bandwidth-Efficient Hybrid Video Streaming Over Mobile Networks”

(Edited by Christian Timmerer)

The streaming of audio and video now accounts for more than 70% of the North American downstream traffic in the peak evening hours on fixed access net-works [1]. Also in mobile networks we can observe a similar situation where the "Real-Time Entertain-ment" traffic (i.e., streaming of audio and video) is by far the most dominant traffic category which accounts for more than 40% of the downstream bytes as shown in Figure 1 [2]. Given this trend there a need for providing streaming solutions that address the special needs for mobile networks. Important aspects in mo-bile video streaming are energy-awareness and band-width-efficiency which are both addressed in the pa-per by Almowuena et al. In this paper, the authors investigate cellular networks supporting video streaming over unicast or multicast. In particular, they point out an existing tradeoff be-tween unicast and multicast. The former leads to higher network load but lower energy consumption of the devices while the latter results in lower network load but higher energy consumption. Therefore, au-thors propose to utilize both delivery schemes concur-rently in order to get the best out of both while reduc-ing their drawbacks. The proposed solution comprises a hybrid multicast-unicast streaming system for which a resource allocation is formulated as a binary integer programming problem and efficient heuristic algo-rithms evaluated using trace-driven simulation. The authors have described the problem statement of resource allocation to be deployed in wireless base stations as NP-Complete and proposed a solution (1) to maximize the average energy saving, (2) which makes sure that only a given amount of network re-sources is consumed, and (3) guarantees that every mobile device receives its allocation window at a fea-sible modulation and coding scheme mode. The prob-lem solver CPLEX is used to implement the optimal algorithm which suffers from an exponential worst-case runtime but can be used as a benchmark. Based on that, the authors provide two efficient algorithms for both the single-cell and multi-cell allocation prob-lem.

The evaluation is done within a simulation environ-ment based on OPNET that demonstrates the near optimality of the proposed algorithms as well as a significant increase of the number of served users and a reduction of the overall energy consumption while imposing a minimal overhead on the cellular network. Additionally, authors simulated a single frequency network to show that the proposed solution increases the number of served mobile users and saves more energy of mobile devices. The proposed solution is applicable to any multicast-capable cellular network but authors considered LTE as an example within their paper. The evaluation includes the following metrics: service ration, spectral efficiency, energy saving, video quali-ty, initial buffering, re-buffering, and channel quality reports overhead. The results clearly indicate the ad-vantages of the proposed solution compared to state-of-the-art approaches and promotes the use of hybrid video streaming to serve the rapidly increasing de-mand of video services over cellular networks.

Almowuena, S.; Rahman, M.M.; Cheng-Hsin Hsu; Hassan, A.A.; Hefeeda, M., "Energy-Aware and Bandwidth-Efficient Hybrid Video Streaming Over Mobile Networks," in Multimedia, IEEE Transactions on , vol.18, no.1, pp.102-115, Jan. 2016

Figure 1. Peak traffic – North America, Mobile

Access [2].



In the future it would be worthwhile to investigate whether the proposed approach can be actually inte-grated within deployments of the existing eMBMS – which has been used as a basis for the authors’ pro-posed solution – in order to see the benefits in a real-world environment. In particular, it would be interest-ing to see an application-level evaluation in the con-text of adaptive HTTP streaming and how the pro-posed approach can be used to effectively save more device energy while at the same time reduce network load. References: [1] C. Timmerer, "Real-Time Entertainment now ac-

counts for >70% of the Internet Traffic", http://multimediacommunication.blogspot.co.at/2015/12/real-time-entertainment-now-accounts.html, December 2015.

[2] Sandvine, "Global Internet Phenomena Report Africa, Middle East, and North America", Sandvine Intelligent Broadband Networks, De-cember 2015.

Christian Timmerer is an As-sociate Professor at Alpen-Adria-Universität Klagenfurt, Austria and his research focus is on immersive multimedia com-munication, streaming, adapta-tion, and quality of experience. He was general chair of WI-AMIS 2008, QoMEX 2013, and ACM MMSys 2016 and has

participated in several EC-funded projects, notably DANAE, ENTHRONE, P2P-Next, ALICANTE, So-cialSensor, ICoSOLE; and the COST Action IC1003 QUALINET. Dr. Timmerer also participated in ISO/MPEG work for several years – notably, in the area of MPEG-21, MPEG-M, MPEG-V, and MPEG-DASH. He is a co-founder of bitmovin and CIO | Head of Research and Standardization. Follow him on http://www.twitter.com/timse7 and subscribe to his blog http://blog.timmerer.com.



Paper Nomination Policy

Following the direction of MMTC, the R-Letter platform aims at providing research exchange, which includes examining systems, applications, services and techniques where multiple media are used to deliver results. Multimedia includes, but is not restricted to, voice, video, image, mu-sic, data and executable code. The scope covers not only the underlying networking systems, but also visual, gesture, signal and other aspects of communication.

Any HIGH QUALITY paper published in Com-munications Society journals/magazine, MMTC sponsored conferences, IEEE proceedings, or other distinguished journals/conferences within the last two years is eligible for nomination.

Nomination Procedure Paper nominations have to be emailed to R-Letter Editorial Board Directors: Christian Timmerer ([email protected]) and Yan Zhang ([email protected]). The nomination should include the complete reference of the paper, author information, a brief supporting statement (maximum one page)

highlighting the contribution, the nominator in-formation, and an electronic copy of the paper, when possible.

Review Process Members of the IEEE MMTC Review Board will review each nominated paper. In order to avoid po-tential conflict of interest, guest editors external to the Board will review nominated papers co-authored by a Review Board member. The reviewers’ names will be kept confidential. If two reviewers agree that the paper is of R-letter quality, a board editor will be assigned to complete the review letter (partially based on the nomination supporting document) for publication. The review result will be final (no mul-tiple nomination of the same paper). Nominators external to the board will be acknowledged in the review letter.

R-Letter Best Paper Award Accepted papers in the R-Letter are eligible for the Best Paper Award competition if they meet the election criteria (set by the MMTC Award Board). For more details, please refer to http://committees.comsoc.org/mmc/rletters.asp



MMTC R-Letter Editorial Board

DIRECTOR CO-DIRECTOR

Christian Timmerer Yan Zhang Alpen-Adria-Universität Klagenfurt Simula Research Laboratory

Austria Norway EDITORS

Koichi Adachi Institute of Infocom Research, Singapore

Pradeep K. Atrey State University of New York, Albany

Xiaoli Chu University of Sheffield, UK

Ing. Carl James Debono University of Malta, Malta

Bruno Macchiavello University of Brasilia (UnB), Brazil

Joonki Paik Chung-Ang University, Seoul, Korea

Lifeng Sun Tsinghua University, China

Alexis Michael Tourapis Apple Inc. USA

Jun Zhou Griffith University, Australia

Jiang Zhu Cisco Systems Inc. USA

Pavel Korshunov EPFL, Switzerland

Marek Domański Poznań University of Technology, Poland

Hao Hu Cisco Systems Inc., USA

Carsten Griwodz Simula and University of Oslo, Norway

Frank Hartung FH Aachen University of Applied Sciences, Germany

Gwendal Simon Telecom Bretagne (Institut Mines Telecom), France

Roger Zimmermann National University of Singapore, Singapore

Michael Zink University of Massachusetts Amherst, USA

Multimedia Communications Technical Committee Officers Chair: Yonggang Wen, Singapore Steering Committee Chair: Luigi Atzori, Italy Vice Chair – North America: Khaled El-Maleh, USA Vice Chair – Asia: Liang Zhou, China Vice Chair – Europe: Maria G. Martini, UK Vice Chair – Letters: Shiwen Mao, USA Secretary: Fen Hou, China Standard Liaison: Zhu Li, USA

MMTC examines systems, applications, services and techniques in which two or more media are used in the same session. These media include, but are not restricted to, voice, video, image, mu-sic, data, and executable code. The scope of the committee includes conversational, presentation-al, and transactional applications and the underlying networking systems to support them.

multimedia communications technical committee ieee...

Documents