GATE Cloud

         Hamish Cunningham,
         Valentin Tablan,
         Ian Roberts,

         University of Sheffield
         http://gate.ac.uk/

Contents

The GATE Family

Experimental apparatus, software infrastructure, and technology transfer vehicle for text analysis.

an architecture
an IDE: GATE Developer: an integrated development environment for language processing components bundled with a widely used Information Extraction system and a comprehensive set of other plugins
a framework: GATE Embedded: an object library optimised for inclusion in diverse applications giving access to all the services used by GATE Developer and more
- used worldwide by thousands of scientists, companies, teachers and students (>30k downloads per year at present, not counting SVN)
- open source (LGPL), 100% java
a web app: GATE Teamware a collaborative annotation environment for factory-style semantic annotation projects built around a workflow engine
- a process: not "get this software and it will revolutionise your life" but "this is how to implement robust and maintainable services"
GATE Cloud: a parallel and distributed service infrastructure running on Amazon EC2
GATE Mímir: (Multi-paradigm Information Management Index and Repository) a scaleable multiparadigm index built on Ontotext's semantic repository family, GATE's annotation structures database plus full-text indexing from MG4J
and finally...
- related tools from Ontotext (OWLIM, KIM, Linked Data endpoints)
- GATE Wiki
- a community

Cloud computing means many things in many contexts. On GATECloud.net it means:

zero fixed costs: you don't buy software licences or server hardware, just pay for the compute time that you use
near zero startup time: in a matter of minutes you can specify, provision and deploy the type of computation that used to take months of planning
easy in, easy out: if you try it and don't like it, go elsewhere! you can even take the software with you, it's all open source
someone else takes the admin load:
- the GATE team from the University of Sheffield make sure you're running the best of breed technology for text, search and semantics
- cloud providers' data center managers (e.g. at Amazon Inc.) make sure the hardware and operating platform for your work is scaleable, reliable and cheap

parallel execution engine of automatic annotation processes + distributed execution of parallel engine
scalability: automatic scaling of processor swarms running on top of AWS EC2
flexibility: parameters configure behaviour, select the GATE application being executed, the input protocol used reading documents, the output protocol used for exporting the resulting annotations, ...
robustness: jobs run unattended over large data sets
- extensively tested and profiled (no memory leaks)
- errors and exceptions that occur during processing are trapped and reported
- if the process crashes (e.g. hardware failure), can be restarted and resumes execution where it left off

A cloud annotation job

something like the facility that the Information Retrieval Facility was trying to set up for IR more generally
host a growing family of experimental system configurations, data sets, results
currently biased heavily towards information extraction (perhaps some mileage in adding more mainstream IR)
persistence and reuse of experimental setups: virtualisation makes it possible to store not just data but the entire compute platform operable for particular experiments or analyses

Lessons

big data is challenging!
the terms "general purpose" and "large scale" don't sit well together
clouds are moving quickly; how to bet on a winner?
academic clouds need to be as stable as AWS, at least at the API level
commercial clouds not yet a replacement for all academic facilities (e.g. the cost of an EC2 VM continuously for 1 year ~= cost of buying the hardware)

Plans

Amazon Web Services limitations
- S3 costs are prohibitively high for multi-TB data sets
- upload and download are slow (quickest way: a box full of discs...)
- move towards free-for-research cloud
- make similar facilities available on academic computing services
need shared repositories for large text collections
- repeatability of experiments
- reducing costs (data on cloud already)
- anywhere availability