How to Connect to GCP Cloud SQL Instances in Cloud Run Servies | by Lynn Kwong | Jun, 2022
Connect to Cloud SQL instances with private IPs in minutes
If you use the Google Cloud Platform (GCP), it’s common to access Cloud SQL in your Cloud Run services. This is supposed to be an easy task because both Cloud SQL and Cloud Run are in the same network. However, when you do it yourself, you may not be able to do it successfully in a short time because many documents online are either outdated or over-complicated. After some trials and errors for several hours, I finally got it to work. I think it’s worth the effort to write down my solution so it can save others some time as well. It’s actually pretty simple if you do it correctly.
Set up Cloud SQL
Since both Cloud SQL and Cloud Run are on the Google Cloud Platform, it’s better to connect by private IP for minimum network latency. Enable the “Private IP” option if it’s not enabled yet. Keep note of the network and private IP address which will be used later.
Create a VPC connector
Virtual Private Cloud (VPC) is a virtual version of a physical network, implemented inside of Google’s production network which can be used to connect all types of resources created on the platform. It’s actually a pretty complex and abstract concept. For now, you only need to understand that Serverless VPC Access allows Cloud Functions, Cloud Run services, and App Engine apps to access resources in a VPC network using their private IPs. Let’s create a VPC connector that can be used by our Cloud Run service to connect to our Cloud SQL instances.
Search for “Serverless VPC access” in the top search box and choose “Serverless VPC access (VPC network)”. Then click “CREATE CONNECTOR” to create a new VPC connector:
Fill in the form as follows:
- Give it a unique name.
- Choose the same region as your Cloud SQL instance and Cloud Run service. This is very important otherwise you cannot connect to the Cloud SQL instance with this connector.
- Choose the same network as your Cloud SQL instance. This is also important if you have multiple networks.
- Choose “Custom IP range” for “Subnet”.
- Specify the IP range to cover the private IP address of your Cloud SQL instance. For this example, it’s set to “10.104.0.0”. The IP range cannot conflict with existing ones.
Now you can click “CREATE” to create the VPC connector. It should be noted that VPC connectors are not free. They are actually pretty expensive to use. Therefore, do remember to delete them when they are not needed anymore.
Configure a Cloud Run service account
The Cloud Run service account, which is Compute Engine default service account by default, should have the Cloud SQL Client role with permissions to connect to Cloud SQL.
On the IAM page, search for “Compute Engine default service account”, then click the pencil button to add the Cloud SQL Client role for it:
Create a Cloud Run service
Finally, let’s create a Cloud Run service and connect it to our Cloud SQL instance with the VPC connector just created. Try to find “Cloud Run” on the GCP platform and click “CREATE SERVICE” to create a new one:
Importantly, choose the same region as your Cloud SQL instance and VPC connector. For a more detailed introduction to how to deploy a Cloud Run service automatically, this post can be helpful.
For demonstration purposes, we would normally select “Allow all traffic” and “Allow unauthenticated invocations”. You can add more fine-grained authentication if needed with IAM.
Now expand the “Container, Variables & Secrets, Connections, Security” section and specify the connections and environment variables. Let’s set the connection first. Click the “CONNECTIONS” tab, then click “ADD CONNECTION” to add a new one. Select the Cloud SQL instance to which you want to connect. Then choose the “VPC connector” created in the previous step. Note that the VPC connector won’t be shown if the region of the connector is different from that of the Cloud Run service being created.
Then we need to create some environment variables for our DB connection which would be used in our Cloud Run service:
Very importantly! We need to specify the private IP address of our Cloud SQL instance as the DB host, not “127.0.0.1” or “localhost”, as shown in some tutorials, otherwise, it won’t work.
The environment variables will be used in the code of your Cloud Run service to connect to the Cloud SQL instance. If you use SQLAlchemy to connect to a MySQL database, the code would be like this:
If needed, you can check the corresponding articles for how to use SQLAlchemy to execute plain SQL queries and create ORM models.
Now you should be able to connect to your Cloud SQL instances from your Cloud Run services. Have a try!
Conclusion
It’s very common that we need to connect to Cloud SQL if we host our backend code on Cloud Run. However, trivial as it may seem, there are quite a few pitfalls that will make the connection fail. In this article, we have covered the general setup procedures and also highlighted the pitfalls that may incur hours of struggling time. The main takeaway from this post is as follows:
- The network of the SQL instance and the VPC connector should be the same.
- The regions of the SQL instance, VPC connector, and Cloud Run service should be the same.
- The private IP of the SQL instance should be used in the Cloud Run service for connection.
Connect to Cloud SQL instances with private IPs in minutes
If you use the Google Cloud Platform (GCP), it’s common to access Cloud SQL in your Cloud Run services. This is supposed to be an easy task because both Cloud SQL and Cloud Run are in the same network. However, when you do it yourself, you may not be able to do it successfully in a short time because many documents online are either outdated or over-complicated. After some trials and errors for several hours, I finally got it to work. I think it’s worth the effort to write down my solution so it can save others some time as well. It’s actually pretty simple if you do it correctly.
Set up Cloud SQL
Since both Cloud SQL and Cloud Run are on the Google Cloud Platform, it’s better to connect by private IP for minimum network latency. Enable the “Private IP” option if it’s not enabled yet. Keep note of the network and private IP address which will be used later.
Create a VPC connector
Virtual Private Cloud (VPC) is a virtual version of a physical network, implemented inside of Google’s production network which can be used to connect all types of resources created on the platform. It’s actually a pretty complex and abstract concept. For now, you only need to understand that Serverless VPC Access allows Cloud Functions, Cloud Run services, and App Engine apps to access resources in a VPC network using their private IPs. Let’s create a VPC connector that can be used by our Cloud Run service to connect to our Cloud SQL instances.
Search for “Serverless VPC access” in the top search box and choose “Serverless VPC access (VPC network)”. Then click “CREATE CONNECTOR” to create a new VPC connector:
Fill in the form as follows:
- Give it a unique name.
- Choose the same region as your Cloud SQL instance and Cloud Run service. This is very important otherwise you cannot connect to the Cloud SQL instance with this connector.
- Choose the same network as your Cloud SQL instance. This is also important if you have multiple networks.
- Choose “Custom IP range” for “Subnet”.
- Specify the IP range to cover the private IP address of your Cloud SQL instance. For this example, it’s set to “10.104.0.0”. The IP range cannot conflict with existing ones.
Now you can click “CREATE” to create the VPC connector. It should be noted that VPC connectors are not free. They are actually pretty expensive to use. Therefore, do remember to delete them when they are not needed anymore.
Configure a Cloud Run service account
The Cloud Run service account, which is Compute Engine default service account by default, should have the Cloud SQL Client role with permissions to connect to Cloud SQL.
On the IAM page, search for “Compute Engine default service account”, then click the pencil button to add the Cloud SQL Client role for it:
Create a Cloud Run service
Finally, let’s create a Cloud Run service and connect it to our Cloud SQL instance with the VPC connector just created. Try to find “Cloud Run” on the GCP platform and click “CREATE SERVICE” to create a new one:
Importantly, choose the same region as your Cloud SQL instance and VPC connector. For a more detailed introduction to how to deploy a Cloud Run service automatically, this post can be helpful.
For demonstration purposes, we would normally select “Allow all traffic” and “Allow unauthenticated invocations”. You can add more fine-grained authentication if needed with IAM.
Now expand the “Container, Variables & Secrets, Connections, Security” section and specify the connections and environment variables. Let’s set the connection first. Click the “CONNECTIONS” tab, then click “ADD CONNECTION” to add a new one. Select the Cloud SQL instance to which you want to connect. Then choose the “VPC connector” created in the previous step. Note that the VPC connector won’t be shown if the region of the connector is different from that of the Cloud Run service being created.
Then we need to create some environment variables for our DB connection which would be used in our Cloud Run service:
Very importantly! We need to specify the private IP address of our Cloud SQL instance as the DB host, not “127.0.0.1” or “localhost”, as shown in some tutorials, otherwise, it won’t work.
The environment variables will be used in the code of your Cloud Run service to connect to the Cloud SQL instance. If you use SQLAlchemy to connect to a MySQL database, the code would be like this:
If needed, you can check the corresponding articles for how to use SQLAlchemy to execute plain SQL queries and create ORM models.
Now you should be able to connect to your Cloud SQL instances from your Cloud Run services. Have a try!
Conclusion
It’s very common that we need to connect to Cloud SQL if we host our backend code on Cloud Run. However, trivial as it may seem, there are quite a few pitfalls that will make the connection fail. In this article, we have covered the general setup procedures and also highlighted the pitfalls that may incur hours of struggling time. The main takeaway from this post is as follows:
- The network of the SQL instance and the VPC connector should be the same.
- The regions of the SQL instance, VPC connector, and Cloud Run service should be the same.
- The private IP of the SQL instance should be used in the Cloud Run service for connection.
Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.