Extending assist for App Engine bundled providers (Module 17)



Posted by Wesley Chun (@wescpy), Developer Advocate, Google Cloud


App Engine initially launched in 2008, offering a suite of bundled providers making it handy for functions to entry a database (Datastore), caching service (Memcache), unbiased process execution (Process Queue), Google Signal-In authentication (Customers), or giant “blob” storage (Blobstore), or different companion providers. Nevertheless, apps leveraging these providers can solely run their apps on App Engine.

To extend app portability and assist Google transfer in direction of its aim of getting probably the most open cloud available on the market, App Engine launched its 2nd-generation service in 2018, initially eradicating these legacy providers. The newer platform permits builders to improve apps to the newest language runtimes, akin to shifting from Python 2 to three or Java 8 to 11 (and at this time, Java 17). One of many main drawbacks to the 1st-generation runtimes is that they are personalized, proprietary, and restrictive in what you should use or cannot.

As an alternative, the 2nd-generation platform makes use of open supply runtimes, that means skill to observe commonplace growth practices, use widespread/identified idioms, and have fewer restrictions of Third-party libraries, and obviating the necessity to copy or “vendor” them together with your code. Sadly, to make use of these newer runtimes, migrating away from App Engine providers have been required as a result of whilst you might improve language releases, there was no entry to bundled providers, breaking apps or requiring full rewrites, making it a showstopper for a lot of customers.

On account of their recognition and the need to ease the improve course of for patrons, the App Engine group restored entry to most (however not all) of these providers in Fall 2021. At present’s Serverless Migration Station video demonstrates learn how to proceed utilization of bundled providers accessible to Python 3 builders.

Exhibiting App Engine customers learn how to use bundled providers on Python 3

Performing the improve

Modernizing the everyday Python 2 App Engine app seems to be one thing like this:

  1. Migrate from the webapp2 framework (not accessible in Python 3)
  2. Port from Python 2 to three, protect use of bundled providers
  3. Optionally available migration to Cloud standalone or related Third-party providers

Step one is to maneuver to a normal Python net framework like Flask, Django, Pyramid, and so forth. Under is a few pseudocode from Migration Module 1 demonstrating learn how to migrate from webapp2 to Flask:

codeblocks for porting Python 2 sample app from webapp2 to Flask
Step 1: Port Python 2 pattern app from webapp2 to Flask

The important thing modifications are bolded within the above code snippets. Discover how the App Engine NDB code [the Visit class definition plus store_visit() and fetch_visits() functions] are unaffected by this net framework migration. The complete webapp2 code pattern will be discovered within the Module 0 repo folder whereas the finished migration to Flask pattern is positioned within the Module 1 repo folder.

After your app has ported frameworks, you are free to improve to Python 3 whereas preserving entry to the bundled providers in case your app makes use of any. Under is pseudocode demonstrating learn how to improve the identical pattern app to Python 3 in addition to the code modifications wanted to proceed to make use of App Engine NDB:

codeblocks for porting sample app to Python 3, preserving use of NDB bundled service
Step 2: Port pattern app to Python 3, preserving use of NDB bundled service
The unique app was designed to work underneath each Python 2 and three interpreters, so no language modifications have been required on this case. We added an import of the brand new App Engine SDK adopted by the important thing replace wrapping the WSGI object so the app can entry the bundled providers. As earlier than, the important thing updates are bolded. Some updates to configuration are additionally required, and people are outlined within the documentation and the (Module 17) codelab.

The NDB code can also be left untouched on this migration. Not all the bundled providers characteristic such a hands-free migration, and we hope to cowl a few of the extra advanced ones forward in Module 22. Java, PHP, and Go customers have it even higher, requiring fewer or no code modifications in any respect. The Python 2 Flask pattern is positioned within the Module 1 repo folder, and the ensuing Python 3 app will be discovered within the Module 1b repo folder.

The quick good thing about step two is the power to improve to a extra present model of language runtime. This leaves the third step of migrating off the bundled providers as non-compulsory, particularly if you happen to plan on staying on App Engine for the long-term.

Extra choices

In case you resolve emigrate off the bundled providers, you are able to do so by yourself timeline. It must be a consideration do you have to ever wish to transfer to fashionable serverless platforms akin to Cloud Capabilities or Cloud Run, to lower-level platforms since you need extra management, like GKE, our managed Kubernetes service, or Compute Engine VMs.

Step three can also be the place the remainder of the Serverless Migration Station content material could also be helpful:

*code samples and codelabs accessible; movies forthcoming

So far as shifting to fashionable serverless platforms, if you wish to break aside a big App Engine app into a number of microservices, contemplate Cloud Capabilities. In case your group has added containerization as a part of your software program growth workflow, contemplate Cloud Run. It is appropriate for apps if you happen to’re aware of containers and Docker, however even if you happen to or your group haven’t got that have, Cloud Buildpacks can do the heavy lifting for you. Listed here are the related migration modules to discover:


Early App Engine customers recognize the comfort of the platform’s bundled providers, and after listening to consumer suggestions, including them again to 2nd-generation runtimes is one other manner we can assist builders modernize their apps. Whether or not upgrading to newer language runtimes to remain on App Engine and proceed to make use of its bundled providers, migrating to Cloud standalone merchandise, or shifting to different serverless platforms, the Google Cloud group goals to supply the instruments to assist streamline your modernization efforts.

All Serverless Migration Station content material (codelabs, movies, supply code [when available]) will be accessed at its open supply repo. Whereas our content material initially focuses on Python customers, the Cloud group is engaged on masking different language runtimes, so keep tuned. At present’s video incorporates a particular visitor to supply a teaser of what to anticipate for Java. For extra video content material, take a look at the broader Serverless Expeditions collection.