graisearch.scss.tcd.ie Open in urlscan Pro
2001:770:10:200:c88d:25ff:fe80:a674  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Toggle navigation
 * Contact
 * Workpackages
 * Team
 * Publications


GRAISEARCH

Use of Graphics Rendering and Artificial Intelligence for Improved Mobile Search
Capabilities

GRAISearch is an Industry-Academia Partnerships and Pathways project (IAPP)
aiming at transferring knowledge from Academia to Industries. It is also a
support for training and career development of researchers (Marie Curie).

The principal aim of this project is to develop and apply revolutionary graphics
rendering and artificial intelligence (AI) methods to an existing social media
search engine platform thereby creating ground breaking mobile search
capabilities with significant online commercial potential.

Technologies from research in two academic institutions (INSA de Lyon (LIRIS UMR
CNRS 5205) and Trinity College Dublin (TCD)), will be transfered into the
products of Tapastreet LTD, the Dublin based Web startup.


    


CONTACT

Julie Connelly
Project Manager
graisearch@scss.tcd.ie

Rozenn Dahyot
Project Coordinator

Project start: 01/02/2014

Project end: 31/01/2018

Funding agency: EU Marie Curie Actions

Project Websites: graisearch.eu and Cordis website

Tweets by @graisearch




WORKPACKAGES


WP1: VIDEO SUMMARISATION

Workpackage Leader: TCD

This package aims at developing video summarization algorithms (vsa) for amateur
social media video to display local event highlights as they occur anywhere in
the world. The first brute force strategy to create a gif summary by
down-sampling videos, will look at the different resolutions that could be
proposed to the user with the requirement that the information displayed is of
good quality for a quick understanding of the content of the input video.
software 1.0 will be used to process a database of videos and create short
summaries at different resolutions. The analysis of the low level content (e.g.
motion, colour) of the video can be performed with metrics defined in
information theory. We propose to create a smart software prototype for creating
summaries that will use these metrics for measuring automatically the content of
videos. These metrics will be used to automatically select what are the images
in the video that should be retained to be part of the summary, and also this
will help in selecting the best spatial resolution automatically. Some artifacts
(hand shake, blur, and occlusion) that often occur in amateur videos will be
dealt with to improve the quality of the summaries.


WP2: 3D SCENE RENDERING

Workpackage Leader: TCD

WP2 aims at developing automatic local 3d scene rendering algorithms (sra)
leveraging public geo-located social media photos of a particular location.
merging several images or videos to create an augmented image can be a step
further towards creating a good quality summary. this work package led by tcd
will look at merging several images and/or videos recorded at the same place at
the same time for creating a 3d rendering of a scene. using location information
embedded with the input images and videos, and potentially using additional 3d
content available (e.g. google maps in 3d), this work package will look at
computing local descriptors in the images, suitable for image stitching and 3d
reconstruction, but also for image classification (wp5). we propose to use a
modeling based on the generalised relaxed radon transform (gr2t) to estimate a
probability density function of the 3d location and colour. an animated gif will
then be created by moving a virtual camera into the scene and a perceptual
testing using questionnaires and eye tracking technique will be used to assess
the quality of the rendering. the path of the virtual camera will be
automatically chosen such that the summary is both informative, and visually
pleasing. metrics from information theory will be used to assess the information
content of the summary.


WP3: TRAJECTORY MINING

Workpackage Leader: INSA

Mathematical methods and prototypes for mining and predicting local communities
workflows through contextualized trajectory pattern mining applied to social
network data from Tapastreet. For that, the following sub-tasks are considered:
(i) community detection, (ii) places of interest (POI) characterization, and
finally (iii) constrained-based mining of contextualized trajectories. The goal
is to provide a valuable input for WP5 which entails making recommendations to
social media end users w.r.t. their social trajectories and context, the
recommendation could be a breaking news events (whose detection is handled in
WP4) and is triggered by a fix on the persons location.


WP4: EVENT DETECTION

Workpackage Leader: INSA

Develop a computing solution for (i) event detection and (ii) sources of trust
identification in geolocalized social media data streams. It is a centre piece
between WP3 and WP5: Geo-localized breaking events are detected and recommended
to a user entering (or predicted to enter in) the corresponding location. For
task (i), we design/use data-mining techniques to detect/predict unexpected
events/patterns in data streams in presence of concept drift. We propose to
model task (ii) as an original problem of temporal dependency discovery between
some topics from different social media and bring an algorithmic solution
through a graph-mining algorithm.


WP5: RECOMMENDATIONS

Workpackage Leader: INSA

Identification of a recommendation strategy and the design of a recommender
prototype for geo-located social media users in a geo-local context. Thanks to
the results of WP3 and WP4, we make use of user trajectories stratified by
demography, characterized points of interest, and trusted breaking news. Here we
close the loop on WP3&4 who's learnings are applied and built into this
recommender system prototype. The strategy will be based on real data supplied
by Tapastreet, expertise from Tapastreet's machine learning department and
knowledge and expertise from INSA de Lyon.


WP6: IMPLEMENTATION

Workpackage Leader: Tapastreet

WP6 is to develop strategies and methods for implementation of video
summarisation and automated 3d scene rendering into tapastreets social media
search engine platform. Research effort in wp6 is led by tapastreet and will be
looking at what strategy (e.g. cloud computing, parallel processing, gpu
processing, etc.) can be used for a fast reliable implementation of wp1 & wp2
into the tapastreet platform. video summarization will need to be processed
rapidly, and therefore some solutions may be better adapted than others. a
hierarchical approach can be considered where about simple summaries are
proposed and then replaced when the smarter ones becomes available. beside the
strategy for processing the information in a timely fashion, this workpackage
will look at storage of the summaries, and easy access via mobile platform.


PROJECT TEAM


JULIE CONNELLY


PROJECT MANAGER, TCD


ROZENN DAHYOT


PROJECT COORDINATOR, TCD


CELINE ROBARDET


INSA, SECONDEE IN TAPASTREET


MARC PLANTEVIT


INSA, SECONDEE IN TAPASTREET


MARIAN SCUTURICI


INSA, SECONDEE IN TAPASTREET


PIERRE HOUDYER


TAPASTREET, SECONDEE IN INSA


MEHDI KAYTOUE


INSA, SECONDEE IN TAPASTREET


ALBRECHT ZIMMERANN


RECRUITED IN INSA


THOMAS VULIN


RECRUITED IN TAPASTREET, SECONDEE IN TCD


CYRIL BOURGES


TAPASTREET, SECONDEE IN TCD & INSA


ABULLAH BULBUL


RECRUITED IN TCD


ZBIGNIEW ZDZIARSKI


TCD, SECONDEE IN TAPASTREET


PUBLICATIONS

JOURNAL ARTICLES

 * Social Media based 3D Visual Popularity
   A. Bulbul & R. Dahyot, Computer & Graphics, 2017
   [DOI:10.1016/j.cag.2017.01.005]
 * Populating virtual cities using social media, A. Bulbul & R. Dahyot, computer
   animation and virtual worlds journal 2016, [DOI:10.1002/cav.1742].
   Also presented at computer animation and social agents (casa) conference by
   A. Bulbul, Geneva Switzerland, May 2016.

CONFERENCE PROCEEDINGS

 * Deep shape from a low number of silhouettes
   X. Di, R. Dahyot, M. Prasad, eccv workshop geometry meets deep learning,
   Amsterdam, 9th october 2016 (presenter: X. Di)
   [DOI:10.1007/978-3-319-49409-8_21]
 * Profiling Users of the Velo'v Bike Sharing System
   A. Zimmermann, M. Kaytoue, M. Plantevit, C. Robardet, J.-F. Boulicaut.
   Proceedings of the 2nd International Workshop on Mining Urban Data co-located
   with 32nd International Conference on Machine Learning (ICML 2015), Lille,
   France, July 11th, 2015 (MUD@ICML 2015), pages 63-64, [PDF].
 * Gazouille: Detecting and Illustrating Local Events from Geo-localized Social
   Media Streams
   P. Houdyer, A. Zimmerman, M. Kaytoue, M. Plantevit, C. Robardet, J. Mitchell.
   in European Conference on Machine Learning and Principles and Practice of
   Knowledge Discovery in Databases (ECML PKDD 2015), Part III, LNAI 9286
 * Social Media based Up-to-Date 3D Modeling and Visualization
   A. Bulbul and R. Dahyot, Conference on Visual Media Production, London,
   November 2015. DOI:10.1145/2824840.2824860, (Presenter: A. Bulbul)[PDF]
 * 3D Reconstruction of Reflective Spherical Surfaces from Multiple Images
   A. Bulbul, M. Grogan and R. Dahyot, Irish Machine Vision and Image Processing
   conference, pages 19-26, (Permanent link to full book:
   http://hdl.handle.net/2262/74714) ISBN 978-0-9934207-0-2, August 2015.
   (Presenter: A. Bulbul) [PDF]
 * L2 Registration for Colour Transfer
   M. Grogan, M. Prasad and R. Dahyot, European Signal Processing Conference
   (Eusipco), ISBN 978-0-9928626-4-0, Nice France, September 2015.
   DOI:10.1109/EUSIPCO.2015.7362799 (Presenter: R. Dahyot) [PDF]
 * L2 registration for Colour Transfer in Videos
   M. Grogan and R. Dahyot, short paper in Conference on Visual Media
   Production, London, November 2015. DOI:10.1145/2824840.2824862 (Presenter: M.
   Grogan) [PDF]
 * Information visualisation for social media analytics
   R. Dahyot, C. Brady, C. Bourges and A. Bulbul, International Workshop on
   Computational Intelligence for Multimedia Understanding, Prague, Czech
   Republic, 29-30 Oct. 2015. DOI:10.1109/IWCIM.2015.7347082 and some Code on
   GitHub (Presenter: R. Dahyot) [PDF]
 * Triggering patterns of topology changes in dynamic graphs
   M. Kaytoue, Y. Pitarch, M. Plantevit, C. Robardet. in Advances in Social
   Networks Analysis and Mining (ASONAM), 2014 IEEE/ACM International Conference
   on ,pp.158,165, 17-20 Aug. 2014 [DOI:10.1109/ASONAM.2014.6921577] [PDF]
 * On summarising the 'here and now' of social videos for smart mobile browsing
   Z. Zdziarski, C. Bourges, J. Mitchell, P. Houdyer, D. Johnson, and R. Dahyot,
   International Workshop on Computational Intelligence for Multimedia
   Understanding, Paris, 1-2 Nov. 2014. DOI:10.1109/IWCIM.2014.7008797
   (Presenter: Z. Zdziarski) [PDF]
 * An Architecture for Social Media Summarisation
   Z. Zdziarski, J. Mitchell, P. Houdyer, D. Johnson, C. Bourges and R. Dahyot,
   Irish Machine Vision and Image Processing Conference, Derry-Londonderry,
   Northern Ireland, 27-29 August 2014. http://hdl.handle.net/2262/71411
   (Presenter: C. Bourges) [PDF]
 * Mesh from Depth Images Using GR2T
   M. Grogan and R. Dahyot, Irish Machine Vision and Image Processing
   Conference, Derry-Londonderry, Northern Ireland, pp. 15-20, 27-29 August
   2014. http://hdl.handle.net/2262/71411 (Presenter: M. Grogan) [PDF]
 * GR2T Vs L2E with nuisance scale
   R. Dahyot, International Conference on Pattern Recognition (ICPR), Sweden,
   August 2014. DOI:10.1109/ICPR.2014.662 (Presenter: R. Dahyot) [PDF]

TECHNICAL REPORTS

 * On the cutting edge of event detection from social streams - a non-exhaustive
   suvey (external version).
 * Graisearch and INSA Partner in a nutshell. Seminar at TCD.

GRAISearch - FP7-PEOPLE-2013-IAPP - Grant Agreement Number 612334 - Webmaster:
graisearch@scss.tcd.ie