The main challenges with existing Grid Computing approaches is that they require high technical knowledge and dedicated hardware and software resources. For these reasons, the deployment and operational cost of such grid systems is substantial, preventing their adoption and direct use by non-technical organizations and users, such as biological researchers in a small university lab. Only a relatively few dedicated scientists use the classic grids like Globus; and setting up projects in volunteer computing grids like BOINC is rather centralized and require running a dedicated server infrastrucure.
In contrast to these large-scale grid infrastructures, our goal has been to create a distributed and ad-hoc grid computing platform for scientist to use as part of their work in the biology laboratory.
The goals were to create a grid infrastructure which is:
- Voluntary – biologist can leverage unused resources on computers in the vicinity by offloading computational tasks to them.
- Symmetric – all users can equally submit jobs as well as help execute jobs.
- Dynamic – task allocation is done on an ad-hoc, peer-to-peer fashion with no central control.
- Easy to Use – the deployment and management of the grid should require no special technical skills of staff; the grid is run by biologist themselves. This goes for both submission as well as execution of jobs.
- Context-Aware – the distribution and execution of jobs take into consideration a match between the requirements of a job and the context of an executing machine.
The Mini-Grid Framework [1, 2]. is a java-based programming API and runtime environment for ad-hoc mini-grids utilising peer-to-peer computing in a local area network of personal computers. It applies a market-based scheduling algorithm based on dynamic context-based resource and application modeling [3].. The Mini-Grid Framework enables the construction and the deployment of ad-hoc grids with the following characteristics:
- Symmetric resource usage – The users can contribute their resources to the Mini-Grid environment as well as use the available resource in the environment.
- Smaller scale grid formation – The Mini-Grid environment can be created by combining the power of the computers at an institutional level or at an organizational/institutional level.
- Decentralized task distribution – The Mini-Grid Framework adopts auction strategies for dynamic resource discovery and selection.
- Resource and task modeling – We have used ontology based context model to describe resource capabilities and to model resource requirements of an application.
- Application specific quality parameters – The Mini-Grid Framework supports specification of application specific quality parameters for schedul- ing.
- Ease of deployment – The Mini-Grid environment can be setup with minimal configuration and installation effort.
The Mini-Grid was integrated with the CLC Bio workbench and is developed as part of the Mini-Grid Project. This was work done together with Neelanarayanan (Neela) Venkataraman and the Mini-Grid Framework and its implementation, evaluation, and deployment in a biology lab is documented in his PhD thesis [2].
References
[1] ![[pdf]](https://www.bardram.net/wp-content/plugins/papercite/img/pdf.png)
![[doi]](https://www.bardram.net/wp-content/plugins/papercite/img/external.png)
J. E. Bardram and N. Venkataraman, “The Mini-Grid Framework: Application Programming Support for Ad-Hoc, Peer-to-Peer Volunteer Grids,” in Advances in Grid and Pervasive Computing: 5th International Conference, GPC 2010, Hualien, Taiwan, May 10-13, 2010. Proceedings, P. Bellavista, R. Chang, H. Chao, S. Lin, and P. M. A. Sloot, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, p. 69–80.
[Bibtex]
J. E. Bardram and N. Venkataraman, “The Mini-Grid Framework: Application Programming Support for Ad-Hoc, Peer-to-Peer Volunteer Grids,” in Advances in Grid and Pervasive Computing: 5th International Conference, GPC 2010, Hualien, Taiwan, May 10-13, 2010. Proceedings, P. Bellavista, R. Chang, H. Chao, S. Lin, and P. M. A. Sloot, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, p. 69–80. [Bibtex]
@Inbook{gpc2010:bardram,
author="Bardram, Jakob E.
and Venkataraman, Neelanarayanan",
editor="Bellavista, Paolo
and Chang, Ruay-Shiung
and Chao, Han-Chieh
and Lin, Shin-Feng
and Sloot, Peter M. A.",
title="The Mini-Grid Framework: Application Programming Support for Ad-Hoc, Peer-to-Peer Volunteer Grids",
bookTitle="Advances in Grid and Pervasive Computing: 5th International Conference, GPC 2010, Hualien, Taiwan, May 10-13, 2010. Proceedings",
year="2010",
publisher="Springer Berlin Heidelberg",
address="Berlin, Heidelberg",
pages="69--80",
abstract="Biological scientists work with bioinformatics algorithms that are either computational or data intensive in nature. Distributed platforms such as Grids and Peer-to-Peer (P2P) networks can be used for such algorithms. Classical Grid Computing platforms are available only to a restricted group of biologist, since they are expensive, and require skilled professionals for deployment and maintenance. Projects deployed using volunteer computing systems require a high visibility. The alternative, peer-to-peer platform is mainly used for sharing data. This paper presents the Mini-Grid Framework, a P2P infrastructure and programming framework for distribution of computational tasks like bioinformatics algorithm. The framework contributes with concepts and technologies for minimal configuration by non-technical end-users, a `resource-push' auction approach for dynamic task distribution, and context modeling of tasks and resources in order to handle volatile execution environment. The efficiency of the infrastructure has been evaluated using experiments.",
isbn="978-3-642-13067-0",
doi="10.1007/978-3-642-13067-0_11",
url="https://doi.org/10.1007/978-3-642-13067-0_11"
}[2] N. N. Venkataraman, “The Mini-Grid Framework: Application Programming Support for Ad hoc Volunteer Grids,” PhD Thesis, 2013.
[Bibtex]
N. N. Venkataraman, “The Mini-Grid Framework: Application Programming Support for Ad hoc Volunteer Grids,” PhD Thesis, 2013. [Bibtex]
@phdthesis{phdthesis:neela,
title = "The Mini-Grid Framework: Application Programming Support for Ad hoc Volunteer Grids",
abstract = "To harvest idle, unused computational resources in networked environments, researchers have proposed different architectures for desktop grid infrastructure. However, most of the existing research work focus on centralized approach. In this thesis, we present the development and deployment of one such infrastructure, called the Mini-Grid Framework for resource management in ad hoc grids using market-based scheduling and context-based resource and application modeling. The framework proposes peer-to-peer architecture that supports several futures: ease of deployment, decentralized task distribution, small scale ad hoc grid formation, and symmetric resource. Furthermore, users can model and specify non-performance based parameters that influence resource allocation. We evaluated the framework through simulation experiments at the IT University of Copenhagen (ITU), as well as a pilot deployment at the Interdisciplinary Nanoscience Center (iNano), Aarhus University. For the simulation experiments we used an application that calculates prime numbers between a given range, and another application that searches for a key in a large data set. In the simulation experiments, we studied the technical performance and overhead of the Mini-Grid Framework and compared its performance with other relevant systems. For the pilot deployment, we have integrated a parallelized version of the Basic Local Alignment Search Tool (BLAST) algorithm and the RNA secondary structure prediction algorithm developed by iNano research center with the Mini-Grid Framework. These algorithms have been developed on top of our framework through an integration of the framework with the CLC Bio Workbench, a software suite for bioinformatics algorithms. The pilot deployment studied the resource participation, the deployment efforts needed, and the performance of the framework in a real grid environment. The main contribution of this thesis are: i) modeling entities such as resources and applications using their context, ii) the context-based auction strategy for dynamic task distribution, iii) scheduling through application specific quality parameters, iv) the definition of an extensible API for ad hoc grid formation and v) enabling symmetric resource participation. The Mini-Grid framework has been designed and developed as proof-of-concept. The Mini-Grid framework has been evaluated using LAB deployment at ITU, and has been deployed at iNano research center using real-life application.",
keywords = "Peer-to-peer Grids, Context-aware Scheduling, Context Modelling, Volunteer Grids, Application Specific Quality of Service",
author = "Venkataraman, Neelanarayanan Narayanan",
year = "2013",
isbn = "978-87-7949-292-9",
}[3] N. Venkataraman, “A Context-aware Task Scheduling in Ad-hoc Desktop Grids,” Procedia Computer Science, vol. 50, iss. Supplement C, pp. 653-662, 2015.
[Bibtex]
N. Venkataraman, “A Context-aware Task Scheduling in Ad-hoc Desktop Grids,” Procedia Computer Science, vol. 50, iss. Supplement C, pp. 653-662, 2015. [Bibtex]
@article{pcs2015:neela,
title = "A Context-aware Task Scheduling in Ad-hoc Desktop Grids",
journal = "Procedia Computer Science",
volume = "50",
number = "Supplement C",
pages = "653 - 662",
year = "2015",
note = "Big Data, Cloud and Computing Challenges",
issn = "1877-0509",
doi = "10.1016/j.procs.2015.04.099",
url = "https://doi.org/10.1016/j.procs.2015.04.099",
author = "Neelanarayanan Venkataraman",
keywords = "context information management, desktop grid computing, resource description, quality of service"
}