research and big data, a virtuous circle of technical progress

The explosion in the number of data produced and the processing capacity which accompanies it opens up previously unexpected prospects for all areas of research. To understand the change in scale of data production, Eric Shmidt, Google’s boss said "every other day we produce as much information as we have generated since the dawn of civilization until 2003”.

IDC estimates that the volume of the digital universe will double every year from 4400 billion gigabytes to 44 billion in 2020. It took 10 years to achieve the first sequencing of human DNA. Nowadays specialized companies are able to do this in five days.

Most states have grasped the importance of the subject and are giving themselves the means to enter the race for big data. Public research programs are funded in a meaningful way. The United States endowed the Big Data Research and Development Initiative with a budget of $200 million. France has released 25 million Euros for Big Data research. Big data is one of the priorities of the 7th R & D framework program of the European Union.

Scientific projects are a great boost for research in data processing. The astrophysics programs are a good example. In comparison, during the eight years of the Sloan Digital Sky (200-2008), 140 terabytes of images were collected. Once set up in 2020, the Large Synoptic Survey Telescope (LSST) will take five days to achieve the same result. According to LSST astronomers, current technologies would take more than 10 years to analyze the images and data produced by the program. The main lines of research will concern the storage, exploitation and sharing of this information, as well as the collaboration of the entities involved.

In France, the Mastodons project of CNRS began work in 2012. It supports interdisciplinary projects that will study the algorithms, methodologies and infrastructures necessary for storing, processing, analyzing, visualizing but also protecting mega data.


CASD, a French solution for sharing data between private stakeholders and researchers

Carried out by Kamel Gadouche, this project for the sharing of data for research purposes was born in 2010. Its objective was to open access to INSEE data more widely to researchers but also to certain ministries and Stakeholder data, particularly in the areas of banking, insurance and energy. Using sensitive and confidential data, it had to meet very high security constraints. Two challenges have also arisen to its bearer: convince on the one hand private data-producing companies to share them with maximum security, and on the other hand researchers to use this program thanks to ergonomics and simplicity of use. Two challenges were identified as RTE, Generali and La Poste quickly became contributors. Designed before the Big Data surge, CASD has acquired since 2013 the ability to process mega data with the addition of tools like Hadoop or Spark.

Concretely, the CASD is a secure space for consultation and data processing. Publication of results is subject to strict rules and mutual agreements between data producers (who decide the conditions of publication) and their users. The CNIL is also involved in the process.
CASD is accessible through a casing coupled with a personalized bimetric smart card for each user. “Its installation must meet strict security requirements and a demanding infrastructure, contractually binding each stakeholder. A “bubble” system creates a complete insulation of the limp and its user, operating in closed circuit, without contact with the outside from the moment the user entered the platform. (Source: Big Data directory 2015-2016) This technical choice was motivated by a calculation of the maintenance costs: a software solution presented too many risks and indirect costs. The solution ensures a very high level of safety, the 350 boxes (for 1000 users) require only 4 technicians dedicated to their maintenance.


CASD is therefore a great French success that is exported: it is working with the European Union to create a common infrastructure for data sharing.

More articles

  • very high speed ​​broadband program: France is struggling to connect all its municipalities

    In 2015, there were still 238 French communes without mobile phone coverage and 2,200 communes without access to 3G or 4G. In these areas, digital technology can help access many services and revitalize territories. In its report of January 2017, […]

  • big data and energy savings: new energy efficiency

    For around ten years, home automation products have been enabling individuals and companies to take control of their homes and offices. Lighting, heating, opening: everything is programmed and controlled remotely. This is definitely a long-term trend. EDF launching its new […]

  • big data and smart farming : farmers are connected

    Managers, caregivers, operators, technicians and farmers are able to do 1000 things at once and that too often single-handedly. The contribution of technology in measuring and monitoring crops and animals has proved to be highly beneficial in producing more and […]

  • the future factory and the big data

    Indeed, 96% of companies intend to use the internet of objects before 2018. The market for industrial software already represents 8 600 billion euros for an annual growth of more than 8%. Finally, “industry 4.0 concerns a large number of […]

  • finance, banking and insurance: how big data is changing these sectors?

    Indeed the Knowledge of markets and time control are the keys to efficient asset management. The slightest geopolitical or commercial event can have immediate impacts on the markets and the price of a share. Also, the United Overseas Bank (UOB) […]

  • big data and health: mega data for personalized medicine

    Big Data and Health, a growing partnership However, key figures in that sector show that it is in full expansion: since 2010, 200 companies specializing in big data and health have been created, in the European Union, 5,5 billion Euros […]

  • the internet of objects and big data, a winning pair

    It’s the extension of the network to objects via sensors. Transport, home automation, health or even insurance, many sectors have strong growth potential thanks to this technological innovation. In 2010, the number of mobile devices was estimated at 5 billion […]

  • Duccio Piovani at ComplexCity

    An edition around the theme of ComplexCity Every year, SICC organise a tutorial series in order to explore the emergence of new areas of research in which the design, the assessment and the control of non-linear and complex systems play an […]