Introducing new Cloud Source Repositories
Russell Wolf
Product Manager
At Google, we spend a lot of time each day working with code. As Google has grown and the code base has increased in complexity, Google engineers have built a set of code tools to help our developers stay happy and productive each day. One particularly essential tool is code search, which is well-loved by Google engineers, and used by most engineers here multiple times a day to improve their productivity.
We’re pleased to bring code search to you with the newly revamped Cloud Source Repositories in beta availability. It features an entirely new user interface and semantic code search capabilities. Cloud Source Repositories is powered by the same underlying code search infrastructure that Google engineers perform their code searches on every day. You get super fast results and save a lot of time finding that piece of code that you know you’ve seen before. The code search in Cloud Source Repositories uses the same document indexing and retrieval technologies that are used for Google Search, making it work very quickly. Using this code search can improve developer productivity, whether you host your code in Cloud Source Repositories or mirror your code from the cloud versions of GitHub or Bitbucket.
Speeding up code search for better development
As developers ourselves, we know how frequently we need to search code. One developer case study found that programmers conduct an average of five search sessions with 12 total queries each workday.
These search queries are often targeted at a particular code location, and programmers are typically looking for code with which they are somewhat familiar. Programmers are generally seeking answers to questions about:
what code does,
where is code instantiated,
why code is behaving a certain way,
who was responsible for an edit and when it happened,
and how to perform a task.
There are some common code search challenges we’ve encountered. Here are a few of those, along with how Cloud Source Repositories handles them.
You want to search across all the code at your company, but there are a lot of repositories with only a few stored locally—which aren’t up-to-date with the versions on the server. Storing code locally isn’t a great option, and your computer’s search tools aren’t very powerful. When you’re using Cloud Source Repositories, the default branches of all repositories are always indexed and up-to-date. It’s simpler and faster to search across all the code you have access to for a particular file/class/function, rather than hunting for the code you have stored locally.
You’re looking for code that performs a common operation that is used by other people in the company. With Cloud Source Repositories, you can do a quick search and find that code. By discovering and using the existing solution rather than reinventing a new solution, you’ll save time, avoid introducing bugs and keep a healthier code base by not adding unnecessary code that has to be maintained over time.
You don’t remember the right way to use a common code component like an event handler. With Cloud Source Repositories, you can enter a query and search across all of your company’s code for examples of how that event handler has been used successfully by other developers. You can write the code correctly the first time.
You discover an issue with your production application. It reports a specific error message to the server logs that reads “User ID 2503295 not found in PaymentDatabase.” You can perform a regular expression search for “User ID .* not found in PaymentDatabase” and instantly find the location in the code where this error was triggered. Then you can get a fix deployed to production for users. Rich regular expression matching allows an easy way to find usages, definitions and sample code, and helps for refactoring.
Search across all your code
Bringing Google’s code search capabilities to Cloud Source Repositories means you’ll get the benefits of Google Search technology for your code base searches.
One key benefit is that now all owned repositories that are either mirrored or added to Cloud Source Repositories can be searched in a single query. This works whether you have a small weekend project or a code base the size of Google’s. And it’s fast: You’ll get the answers you need super quickly—much faster than previous functionality—so you can get back to writing code. And indexing is super fast, too, so the time between new code being added and being available means you’re always searching up-to-date code. Cloud Source Repositories respects all identity and access management (IAM) permissions, so users won’t see any code in search that they shouldn’t have access to, and there are no additional permissions to set up.
In addition, Cloud Source Repositories can search across thousands of different repositories with a single query. You can search for either files or code within files in the default branch of your repository. Cloud Source Repositories also has a semantic understanding of the code, which means that the search index identifies which parts of your code are entities such as classes, functions, and fields. Since the search index has classified these entities, your queries can include filters to limit the search to classes or functions. It also allows for improved search relevance by ranking important parts of code like classes, functions, and fields higher. By default, Cloud Source Repositories allows powerful search patterns using RE2 regular expressions so you can find the answers you need for even very complex questions. (You can work with individual special characters by using a backslash or an entire string by enclosing it in quotation marks.)
As you type in the search box, you’ll get suggestions for matching results (shown below). For Java, JavaScript, Go, C++, Python, TypeScript and Proto files, you’ll see result suggestions indicating whether the match is an entity such as a class, method, enum, or field.
Learn more about searching through your code in Cloud Source Repositories.
How Google code search works
Code search in Cloud Source Repositories uses the same search technologies as Google Search, but optimizes the indexing, algorithms, and result types for searching code. When you submit a query, that query is sent to a root machine and sharded to hundreds of leaf machines. It looks for matches by file names, classes, functions and other symbols, and matches the context and namespace of the symbols. If regular expressions are specified in that query, our code search runs an optimized algorithm to quickly find potential matches for the regular expression. Then, it refines results against the full regular expression to find the actual matches, so the code search can match complex regular expressions very quickly. In addition, Google’s code search looks for relevant snippets of the code around the search to provide additional context for the code match.
Get started with the new Cloud Source Repositories
If you haven’t used Cloud Source Repositories before, you can try it today at no cost with the Google Cloud Platform (GCP) free trial and our generous free tier. When you navigate to Cloud Source Repositories, you’ll see a signup page that will guide you to add your code. You can create a new empty repository or mirror your code from the cloud version of GitHub or Bitbucket. You can easily populate new empty repositories by pushing code from a local machine or writing new code without leaving the browser using the Cloud Shell editor.
If you have existing code stored in Cloud Source Repositories, you’ll be greeted with a new personalized landing page, shown below, that presents a view of all of the repositories you can access across all of your GCP projects. Over time, your personalized landing page will become populated with the recent code you’ve browsed and areas of the code base you’ve favorited.
Once you begin storing your code in Cloud Source Repositories, it’s easy to take advantage of other services from GCP. Some popular integrations are:
Configuring Cloud Build to automatically deploy a new build to App Engine when a commit lands in a branch (View Quickstart)
Using version control for Cloud Functions (View Quickstart)
Inspecting the state of your applications stored in Cloud Source Repositories in real time using Stackdriver Debugger (View Quickstart)
Publishing commit events to Cloud Pub/Sub to integrate with any third-party tool of your choice (View Quickstart)
