Monthly Archives: August 2022

Helpful GCP commands

Posted on August 30, 2022 by Siddharth Jain

2. Login using a service account You shouldn’t do this however. Quoting best practices: Don’t use service accounts during development. During your daily work, you might use tools such as the Google Cloud CLI, gsutil, or terraform. Don’t use a … Continue reading →

Posted in Computers, programming, Software | Tagged , , | Leave a comment

How to iterate over all rows of a table in Oracle db

Posted on August 20, 2022 by Siddharth Jain

Recently I was writing a program to copy data from Oracle to Big Query. This program needs to fetch all rows in a given table (aka a table scan) and then write them to the destination. We should not fetch … Continue reading →

Posted in Computers, programming, Software | Tagged | Leave a comment

Guide to visiting Mt. Rainier

Posted on August 11, 2022 by Siddharth Jain

Of all the fire mountains which, like beacons, once blazed along the Pacific Coast, Mount Rainier is the noblest. John Muir, 1901 Following are the most important things when visiting Mt Rainier: It is hard to simultaneously satisfy the first … Continue reading →

Posted in Travel | Leave a comment

BigQuery Cheat Sheet

Posted on August 10, 2022 by Siddharth Jain

2. List all tables within a dataset 3. List schema of a table aliter: to get as json: 4. Create a dataset: 5. Get job details. Two ways of doing this. Method1: use bq show command: Method2: use REST endpoint. … Continue reading →

Posted in Computers, programming, Software | Tagged | Leave a comment

Measuring performance of BigQuery Java vs Python connector

Posted on August 1, 2022 by Siddharth Jain

While experimenting with Apache Beam, I noticed there is a big difference in the performance of Java vs. Python SDK when reading from BigQuery. See this. So I thought it would be good to measure the performance of the Java … Continue reading →

Posted in Computers, programming, Software | Tagged | Leave a comment