Big Data Project Proposal

Jeremy Gotteland

Data centers require huge investments, rst to build the datacenter, and then powering and cooling the computers require a lot of energy, which ends up having signicant costs and being bad for the environment. But with the rise of the Big Data hype, demand is growing quickly and big companies like IBM are investing massively to build the most performant and scalable data-centers. Nowadays, the number of smartphones, tablets and laptops almost exceed the number of humans. And they are mainly owned by individuals, and those individuals usually do not use all the storage they have on their device. My idea would be to create a software for all of those devices, implementing a peerto-peer network protocol that the user would install and which would lock a part of his unused storage and computer power. Then we could oer the same kind of services as IBM or Amazon oers, but using that new paradigm that would save a lot of money on electricity and building, and hopefully allow us to be competitive on that market (and do some environmental good, as we are at it!). We would have to nd a reward for the user in exchange for that memory though, but the rst goal is to see if such a system would work. I guess it would be like implementing Map/Reduce over IP, but it could also be used for basic storage, and by chunking every le into little pieces and distributing it (and replicates) over the network we would ensure the privacy of the data, and its security as well. The technical challenge would be in designing a peer-to-peer protocol that would allow a potentially huge network of individual devices to compete with the performance of a regular data center. Obviously the mobile devices have lower transmission rates, and common users devices can turn oine at anytime, depending on the user. That will be a huge challenge, but I think mathematical models can be designed and integrated to the protocol or the trackers to make the system robust and make sure the deconnection of one individual doesnt compromise the whole system. The timeline I would propose would be to rst design the protocol, then implement it in a smart-phone application, to eventually adjust the protocol if needed. Then some mathematical models should be designed to improve the service.

Team size would be about 7 people: 2 to design the protocol, 3 to do the software engineering part, and 2 to build the mathematical models. The infrastructure required would be servers similar to BitTorrent trackers, in order to set up a peer-to-peer network.