Category Archives: Computers

Understanding Dataflow – what it can and cannot do

Posted on February 28, 2023 by Siddharth Jain

Google Cloud Dataflow is a popular technology these days to build streaming data pipelines. However it would be useful to remember what it can and cannot do. What Dataflow can do: In above x1, x2, x3 are 3 streaming inputs. … Continue reading →

Posted in Computers, programming, Software | Tagged dataflow | Leave a comment

Can Postgres scale to billions of rows and TB of data?

Posted on February 26, 2023 by Siddharth Jain

It turns out that with proper indexing and partitioning it can! even if the index is so big that it cannot fit in the memory (RAM). To test, I started with a table with 2B rows, 172 cols, 26 partitions … Continue reading →

Posted in Computers, programming, Software | Tagged postgres | Leave a comment

How to make the bloody C# program run?

Posted on February 20, 2023 by Siddharth Jain

Has it happened to you that you wrote a C# program (a Console Application) but it did nothing when you ran it? It happened to me. The issue was this: in the .csproj file I had: change it to: and … Continue reading →

Posted in Computers, programming, Software | Tagged c# | Leave a comment

vi Cheat Sheet

Posted on February 10, 2023 by Siddharth Jain

2. change all text to lowercase: 3. Add string (e.g., *) to end of every line (ref): 4. Add string to beginning of every line: 5. delete last character on every line: 6. Find your vi config file by typing: … Continue reading →

Posted in Computers, programming, Software | Tagged vi | Leave a comment

Performance Comparison of MySQL vs. Postgres on TPC-C Benchmark

Posted on February 8, 2023 by Siddharth Jain

This post describes results of a test to compare the performance of MySQL vs. Postgres on the TPC-C Benchmark for OLTP workloads. The tool used for performance benchmarking was sysbench-tpcc. Note that sysbench-tpcc simulates a TPC-C like workload not exactly … Continue reading →

Posted in Computers, programming, Software | Tagged mysql, postgres | Leave a comment

MySQL vs Postgres

Posted on February 1, 2023 by Siddharth Jain

Pros and Cons of MySQL over Postgres MySQL Postgres – more popular and widely used– more learning resources– uses threads instead of processes for connections– better indexing. secondary indexes point to primary index.– read Uber’s blog why they switched from … Continue reading →

Posted in Computers, programming, Software | Leave a comment

Swift vs. C# Performance

Posted on January 8, 2023 by Siddharth Jain

I am always a sucker for performance. Recently I migrated a .NET GDI+ app to Swift. There were several reasons for it: To my surprise the C# code runs way faster than Swift. The difference is not because of GDI+ … Continue reading →

Posted in Computers, programming, Software | Tagged swift | Leave a comment

Deleting Entities in Bulk from Google Datastore

Posted on January 5, 2023 by Siddharth Jain

The easiest way to do this seems to be using Dataflow. Here is sample Dataflow job to delete all entities of kind foo in namespace bar: As example a job to delete 44,951,022 entities with default autoscaling took 1 hr … Continue reading →

Posted in Computers, programming, Software | Tagged dataflow, datastore | Leave a comment

Performance Benchmarking Various Databases

Posted on December 30, 2022 by Siddharth Jain

This post summarizes results of performance benchmarking some databases. For our test, we use a dataset similar to the Star Schema Benchmark and evaluate the performance on following queries: we created a dataset with 44 M line orders. All databases … Continue reading →