MapReduce for App Engine
Stay organized with collections
Save and categorize content based on your preferences.
Important:
Google has transitioned support and further development of the
Java and Python MapReduce libraries to
the open source community.
The source code and documentation are
available on GitHub
MapReduce is a programming model for processing large amounts of data in a
parallel and distributed fashion. It is useful for large, long-running jobs
that cannot be handled within the scope of a single request, tasks like:
- Analyzing application logs
- Aggregating related data from external sources
- Transforming data from one format to another
- Exporting data for external analysis
App Engine MapReduce is a community-maintained, open source library that is
built on top of App Engine services, including Datastore and Task Queues.
The library is available on GitHub at these locations:
Where to find documentation
The documentation for Mapreduce is available by clicking the wiki icon for the GitHub projects
linked above. However, for your convenience, you can also access the
documentation for both Java and Python MapReduce at the following link:
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-03-05 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-03-05 UTC."],[[["Google has transitioned the Java and Python MapReduce libraries to open-source, with the source code and documentation now available on GitHub."],["MapReduce is a programming model designed for processing large datasets in parallel, making it useful for long-running jobs like log analysis and data transformation."],["App Engine MapReduce is a community-maintained, open-source library built on App Engine services like Datastore and Task Queues."],["The MapReduce library's documentation for both Java and Python is accessible via the GitHub wiki, at the link provided for your convenience."]]],[]]
