ggreg Goto Github PK

• 👋 Hi, I’m @ggreg

I work as a software engineer and also like to code for fun. I live in California with my wife, two kids, and two cats. When not at a computer I enjoy running, rock climbing, playing music, reading about math and science, cooking, and spending time with my family.

• 👀 I’m interested in systems programming: database implementation, compilers, distributed systems, workflow engines, and operating systems. I like to understand how hardware works and be mindful about mechanial sympathy. I enjoy making efficient and performant systems.
• 🌱 I’m currently learning Rust.
• 📫 How to reach me:
  • https://github.com/ggreg
  • https://twitter.com/ggregl
  • https://www.linkedin.com/in/gregleclercq/

Historically my main programming languages have been Python and C/C++. I've done a decent amount of programming in Go, TypeScript, Java, and Haskell. I'm learning Rust and writing a bunch of tools, a compiler (implementing https://craftinginterpreters.com/ in Rust), and a database.

My first computer was an Atari 800. I wrote my first program in Basic on a Thomson MO6 computer and enjoyed programming in Logo in the 80's. Then I got an Atari ST and discovered more complex games and programs. I was interested in indie game and demos.

I started my career in the early 2,000s after the dotcom crash. My first job was on network and computer security at a large ISP. I had the opportunity to work on platforms that served tens of millions of users. I dove into network protocols, how security devices work, and their limitations. I enjoyed writing tools to better test network security devices such as firewalls, Intrusion Protection Systems (IPS), and Web application Firewalls (WAF). This was the time when attacks started to shift from network services to client applications: browsers, email clients, and desktop apps such as Acrobat reader. I've worked on host detection and prevention systems to address that. Then I ended up working on Security Information and Event Management (SIEM). It was still early and these technologies were emerging. I didn't realize at the time how many more opportunities existed to build better security systems. My official title was "architect". I wrote many tools and enjoyed reading the source code of open source software we used but I wanted to build something. I left to join a small company that was building one of the first public cloud.

My first project was to support jumbo frame in the ATA-over-Ethernet client (aoe module in linux) and server (aoeserver module not in the mainstream linux kernel). I enjoyed kernel programming. It was great to actually implement something that run in the linux kernel! I learned so much! Because it was about data and the storage system, it had to be fast and reliable. I worked on performance and testing. I improved performance by 30% by switching to read-copy-update (RCU) instead of the previous locking mechanisms (spinning locks and mutexes).

The storage layer was mostly handling I/O requests for virtual machines. These virtual machines shared a lot of common files. My next project was about implementing a EBS-like block device. We wanted to move above in the network stack and use a protocol over IP. We also wanted something easy to use and deploy. I decided to use NBD and a backend storage server implemented in userspace. I liked kernel programming but I reached diminishing returns in being in the kernel. I could not use some external libraries and it was slower to test. After prototyping different implementations, I developed a event-based server in C based on libev (I preferred it to libevent after testing both) and an append-only block store. It used a bitmap to track the next available block, a tree of blocks for copy-on-write, and compaction to reclaim deleted blocks. I leraned a lot and the block device was efficient. At the same time OpenZFS was open source and it was a great technology. So I decided to bet on something matue, battle-tested at Sun, and now newly available to us for free. As looked at the cloud product we were selling really wanted to provide an API. So with one team mate we implemented the cloud API.

I wanted more ownership and was attracted by the startup world. I wanted to build something from scratch and solve problems in a novel way. I joined an ML startup as a founding engineer and then a web analytics startup. In the latter, I built two product from scratch. The first one ran on bare metal. We made tradeoffs to focus on delivering value while staying lean. The product was innovative and for the size of the company we were processing hundreds of terabytes pretty fast. We built the analytics engine mostly from scratch in Python and some specific libraries in C/C++. I wrote a log parser to process the tens of terabytes of logs some customer sent every day. I was efficient. It worked. We built the second product on AWS. It was in 2012-2013 and AWS wasn't at the maturity it is nowadays. There was no terraform. The only hashicorp product was Vagrant. There was no Docker. We did blue/green deployments with Ansible and define infrastructure as code. We were processing a lot of data from our logs and custom web crawler. We define pipelines in Python and didn't use a framework. We tested Hadoop but did not want to be overwhelmed operating it. I developed a workflow engine based on Amazon SWF. It's similar to https::/temporal.io. We define pipelines as code. They were resilient and we could scale their execution. One time we need to backfill data and ran thousands of worker nodes successfully.

In 2015, I moved to the US and joined the Presto (also known as Trino after Presto's creators left) team at Meta. It was a fantastic experience. I worked at a immense scale and was part of building one of the largest and fastest data infrastructure in the world.

I now work on observability at Airtable.