When I first started learning about Privacy-Enhancing Technologies (PETs) and privacy-preserving machine learning (PPML), all of the content seemed written for graduate-level computer science students, featuring discussions of complex algorithms and equations from domains such as information theory and cryptography. With the help of an undergraduate mathematics degree and some friends I made in different open-source groups, I was able to build a foundation of knowlege in privacy technologies, but it was clear that most PETs and PPML content was not made for self-study.¹

When I joined the xD team of the US Census Bureau, I unexpectedly assumed the role of the go-to privacy person on the team. I found myself answering questions about privacy attacks, federated learning, etc. and putting together ad-hoc documentation for colleagues and stakeholders to review. And once I did this a few times, a few overlapping thoughts occurred to me:

• There's clearly an underserved market for technical privacy content, especially for beginners
• I have a background both in web development and privacy-preserving machine learning
• I've spent several years developing privacy-preserving software, including documenting and demonstrating how it works to other engineers, data scientists, and policy experts

It seemed obvious that I should start a business focused on creating beginner-friendly content to teach others how PETs and other privacy technologies work.

And that's exactly what I'm excited to launch today: https://privacytechcourses.com/.

Read on A Marvelous Balance »

In this post I'd like to share some examples of my own path into an open-source community. One of the great things about OS is that by its very nature most everything is open and available to view on the internet.

Joining OpenMined

At the height of the COVID-19 pandemic in May 2020 I was laid off from the startup I had been at for the last few years. I had been trying to transition to machine learning engineering for a while, and fortunately I had just been accepted into Springboard's ML engineering program. As I progressed through the course material, I learned about the sub-field of privacy-preserving machine learning, or PPML. Initially my interest in this branch of ML was intellectual as the methods that blend cryptography with machine learning techniques are technically fascinating. However I also quickly began to appreciate how PPML could help address a host of social issues inherent is standard ML and AI practices, such as training datasets that can expose the privacy of individuals by including sensitive data such as medical and financial information.

When I began researching groups working on PPML, I quickly came across the group OpenMined. They are a non-profit open-source group dedicated to building PPML software, and after a quick glance of their website I decided to join their Slack group and enroll in a course they had just launched on privacy and technology.

My First Contribution

At one point while working through the course contents I noticed that a radio button on one part of the course web app wasn't working properly, so I opened an "issues" ticket and documented the bug: https://github.com/OpenMined/openmined/issues/92. Initially I thought that pointing out this issue was sufficient, and having been a developer for several years at this point I knew that OpenMined's developers would appreciate a well-documented bug filing. However I kept ruminating on possible causes for the issue, and a few days later I decided to start working on the problem myself. I asked the project lead if I could be assigned the work, which you can see in the comments on that ticket. Once I implemented a solution - a missing property on a React component - I opened a pull-request: https://github.com/OpenMined/openmined/pull/94, and I made sure to adhere to all of the suggested information in their pull-request template such as documenting how I had tested the code change.

After the bug fix was approved and merged, the project lead asked if I'd be interested in contributing more since they were short-staffed (a common thing for open-source groups and non-profits). After passing a brief phone screen I began joining weekly calls with the frontend engineers working on their course material and volunteering a few hours a week with them.

From Open Source to a New Job Opportunity

Within a few months I scaled back my volunteering with OpenMined as I was busy moving houses and then began a new job contracting with NASA, but I kept following OpenMined's work and reading posts in the Slack group. Fortunately I was able to resume working with them a year later when I was accepted into the initial round of their padawan program to dive deeper into privacy-enhancing technologies (PETs) and the PySyft codebase. I've been working with them ever since! And becoming knowledgable of PETs and PySyft directly led to my current role as an emerging technology fellow at the Census Bureau. One of the key points in my favor during the interview process for this role was that I was able to demonstrate an example of my PETs knowledge with a pull-request on Github for a function to compute a differentially private mean: https://github.com/OpenMined/PySyft/pull/6634¹

Simpler Paths to OS Contributions

Admittedly I had been working for several years as a web developer by the time I began contributing to OpenMined's projects, so I felt comfortable diagnosing and fixing the issue with the React component in the courses codebase. But there are plenty of simpler contributions that could be less intimidating to first-time OS contributors such as fixing or adding to project documentation.

As another personal example, when I was first learning about differential privacy I started reading the great book (Programming Differential Privacy). It's an open source book that's entirely hosted on Github, so when I came across some small misspellings I opened pull-requests to get those fixed:

• https://github.com/uvm-plaid/programming-dp/pull/37
• https://github.com/uvm-plaid/programming-dp/pull/38

Beginning Your OS Journey

Hopefully now the idea of contributing to open source seems less intimidating. I often tell people that there is an OS project for any community and interest, and that the OS communities are looking for people with all skills sets, not just software developers. And while knowledge of git used to be a prerequisite for OS contributions, the user interfaces on platforms like Github and GitLab are constantly improving to lower the barrier to making proposed changes and opening pull requests.

Hopefully this post has sufficiently motivated you to begin your open source journey, and if so I encourage you to find a project of interest! Some quick googling and searches or posting a question on Reddit can often help you find some great codebases. As a newcomer I would look for an active codebase (ideally changes are made on a regular basis with the most recent occurring in the last few days or weeks) with some issues marked as “beginner friendly”, “good for first timers”, or something similar. For example, here are the “Good first issue” tickets in OpenMined's PySyft repository.

If you're a developer who is somewhat comfortable with git, I also suggest reading / watching introductions to how to fork repositories and open pull-requests. Here are just two quick examples but there are lots of great resources just a quick search away:

• My first contribution to open source: Make a fork of the repo
• An Easy Introduction to Open Source, Github Cloning, Forking and Pull Requests

Best of luck in your open-source journey, and don't hesitate to contact me if you have questions!

Read on A Marvelous Balance »

Through the ML engineering program I covered a huge range of topics from reviewing how linear regression works to methods for deploying full-scale ML applications in a production environment. I also completed my end-to-end capstone project that I deployed on this site, and met several other Springboard students and mentors across multiple disciplines. I'm especially grateful to my mentor Dhiraj Kumar, who assisted me through our weekly 1:1 calls by answering all of my questions about the ML engineering curriculum and providing guidance at several stages of my capstone project.

Now that I've completed the program my search for a full-time ML engineering role begins in earnest. If you're looking to hire or know a team that is, I'd be extremely grateful if you could let me know! My email is curtis.l.mitchell AT gmail.com.

Read on A Marvelous Balance »

As I mentioned above, beyond the simplest of sentences multiple translations of one language to another are possible. For instance, the English sentence “Yesterday I went to the store” could be translated into Spanish as “Ayer yo me fui a la tienda.” But since Spanish allows the dropping of subjects, in this case “yo” (because listeners can infer the subject from the form of the verb's conjugation), this sentence could also be “Ayer me fui a la tienda.” Here is where confusion about the best translation can start to arise. For instance, for a native Spanish-speaker which sentence is closer to the original English sentence? And this is still a fairly simple example where the two languages are relatively closely related and the word order in each sentence is the same!

Building Blocks

With the advent of statistical machine translation in the last 20 years and the more recent rise of deep learning applications in NLP, researchers have created many metrics to “score” the accuracy of machine translation models. In order to calculate a numerical value these various metrics use a variety of measures and linguistic features in sentences and entire documents (in NLP, a document is usually referred to as a “corpus”, plural “corpuses”), primarily n-grams and precision/recall, which we'll look at here.

n-grams

In NLP n-grams, where “n” stands for any whole number, are the sets of “n” neighboring words in a text. A “unigram” or 1-gram would be any single word, “bigrams” or 2-grams any two neighboring words, and so forth. Below is a table of n-grams for the opening sentence of Nobel Prize-winning author Yasunari Kawabata's famous 1948 novel, Snow Country: “The train came out of the long tunnel into the snow country.”:

unigrams: [“The”, “train”, “came”, “out”, “of, “the”, “long”, “tunnel”, “into”, “the”, ”snow”, “country”]

bigrams: [“The train”, “train came”, “came out”, “out of”, “of the”, “the long”, “long tunnel”, “tunnel into”, “into the”, “the snow”, ”snow country”]

trigrams: [“The train came”, “train came out”, “came out of”, “out of the”, “of the long”, “the long tunnel”, “long tunnel into”, “tunnel into the”, “into the snow”, “the snow country”]

4-grams: [“The train came out”, “train came out of”, “came out of the”, “out of the long”, “of the long tunnel”, “the long tunnel into”, “long tunnel into the”, “tunnel into the snow”, “into the snow country”]

note: N-grams over 3 are typically written with the number, but the bigger the n-gram the less frequently it is used for most NLP tasks.

precision and recall

Precision is a measure of how well your model performs at detecting members of the class you're looking for. We use the total number of correctly and incorrectly labeled positives in the denominator to represent precision as:

$$ \frac{true:positives}{true:positives + false:positives} $$

Recall (also called “sensitivity”) is similar to precision, wherein the percentage of correct labels that were detected by your model is calculated. Because the total number of true positives and false negatives is the total number of examples, we use that as the denominator and thus recall can be represented as:

$$ \frac{true:positives}{true:positives + false:negatives} $$

Note that each metric discussed here uses a different type of average, the details of which I'll present towards the end of the post along with links to the original papers that introduced each metric.

BLEU

The BLEU score has a strong claim to be the most-well known translation metric, as it's usually the one all other metrics are compared to. It was proposed by Kishore Papineni and his coauthors in a 2002 paper at the Annual Meeting of the Association for Computational Linguistics (ACL). BLEU stands for “BiLingual Evaluation Understudy” and compares a candidate sentence (i.e., the machine-generated sentence in the case of NLP models) to reference sentences that are usually created by human translators. The general idea behind the BLEU score is that it compares the n-grams in both the candidate sentence and reference sentences and counts the number of matches. The positions of the n-grams within each sentence do not matter, and the more matches there are the higher the BLEU score is.

Let's look at some examples to build our intuition. We'll compare a known reference sentence and a hypothesis sentence that's hypothetically the output of a machine translation program, and both sentences will be stored as arrays of tokens representing individual words. We can use the sentence-level BLEU score that comes as part of the NLTK API:

from nltk.translate.bleu_score import sentence_bleu
reference_1 = ['the', 'hungry', 'gray', 'dog', 'ate', 'the', 'tasty', 'treat']
hypothesis_1 = ['the', 'hungry', 'gray', 'dog', 'ate', 'the', 'tasty', 'treat']
score = sentence_bleu([reference_1], hypothesis_1)
score # -> 1.0

note: you may notice that the argument for the reference sentence is inside list brackets. This is because the BLEU methods in NLTK assume there will generally be more than one reference sentence passed as a list.

Not surprisingly, this method returns a perfect score when the two sentences are identical.

What happens when we change one word in the hypothesis sentence (in this case, the second “the” becomes “a”)?

hypothesis_2 = ['the', 'hungry', 'gray', 'dog', 'ate', 'a', 'tasty', 'treat']
score = sentence_bleu([reference_1], hypothesis_2)
score # -> 0.5946035575013605

We can see a dramatic drop in the score here. It won't be intuitive based on the score value, but the BLEU score is calculated by comparing the precision of n-grams in the hypothesis and reference sentences. Another thing to note is that by default the BLEU methods in NTLK compare 1- to 4-grams. If we were to only compare unigrams and bigrams by specifying their weighting, our score would be significantly boosted:

score = sentence_bleu([reference_1], hypothesis_1, weights=(0.5, 0.5))
score # -> 0.7905694150420949

More concretely, the BLEU score is comparing the precision and recall of the hypothesis sentence versus the reference sentences. Assuming we're only looking at the unigrams of each sentence, the BLEU score tests to see how many of the words in the reference sentence (8 words total) appear in the hypothesis sentence (7, since the second “the” has become “a”). This would give a score of 7/8 = 0.875.

When computing the BLEU score for an entire corpus, the precision scores of individual sentences are combined using the geometric mean, along with a penalty applied when the length of the reference corpus is shorter is less than or equal the length of the hypothesis sentence.

Although the BLEU score was innovative when introduced and creates scores between 0 and 1 that can be easy to reason about, compared to more recently-introduced methods it can seem somewhat simplistic to use a method that only compares the number of overlapping n-grams.

NIST

In 2002 DARPA (The US Defense Advanced Research Projects Agency) commissioned NIST (the US National Institute of Standards and Technology) to evaluate and create their own machine translation metric based on BLEU. The result was a method similar to the BLEU score but with a few minor changes such as increased weighting for less-common n-grams and a reduced scoring penalty for shorter sentences (known as the “brevity penalty”):

from nltk.translate.nist_score import sentence_nist
score = sentence_nist([reference_1], hypothesis_1)
score # -> 3.0357142857142856

and with the second hypothesis sentence:

score = sentence_nist([reference_1], hypothesis_2)
score # -> 2.642857142857143

Notice that the NIST scores will tend to be greater than one. This is due to the differences between the NIST and BLEU methods given above, plus the fact that the NIST method uses the arithmetic mean of the n-gram overlaps while BLEU uses the geometric mean. See the final section below for details on the differences.

Although the NIST does have slight improvements over BLEU due to the higher weighting of uncommon n-grams and the different calculation of the brevity penalty, those same features make the potential range of score values more difficult to reason about.

METEOR

Another popular translation evaluation metric is METEOR, which stands for “Metric for Evaluation of Translation with Explicit ORdering.” METEOR was designed explicitly to make up for some of the shortcomings of BLEU, including a focus on sentence-level translations as opposed to BLEU's focus on translations at the corpus level. Additionally, METEOR uses the harmonic mean (described in the mathematical details section below) on unigram precision and recall with a greater weighting for recall.

Let's compute the METEOR scores using the same example sentence we used for the BLEU examples, noting that the meteor_score method expects sentences as whole strings instead of an array of tokens as with the BLEU and NIST methods:

from nltk.translate.meteor_score import meteor_score
reference_1 = 'the hungry gray dog ate the tasty treat'
hypothesis_1 = 'the hungry gray dog ate a tasty treat'
score = meteor_score([reference_1], hypothesis_1)
score # -> 0.840561224489796

note: The NLTK meteor_score method takes full sentences as strings (or a list of sentences in the case of reference sentences) for its arguments instead of a list of tokens.

We can see that the METEOR score is significantly higher here versus that of the BLEU score (it was about 0.5946) on the same pair of sentences. This reflects one of the goals of the METEOR metric to more closely mimic human judgement versus the BLEU score, as most people would agree that the sentences are overall very similiar, i.e. that changing “the” to “a” in the translation would be a minor error.

Because of the increased weighting for recall, and the use of the harmonic mean which works better than the arithmetic or geometric means for situations where an average of weights is desired, METEOR is a great choice for machine translation models that is relatively simple to calculate (versus more modern methods) and remains easy to reason about. This is especially true for sentence level translations and these are the reasons I chose to use the METEOR score when evaluating my own translation model that I'll discuss in a future post.

Conclusion

As we've seen from the variety of machine translation evaluation metrics, there's no single method for determining translation accuracy. Depending on which properties of a sentence or corpus are measured, such as unigram recall rate or the weighting of the rareness of particular n-grams, there are a variety of metrics to choose from. On the other hand each metric provides some calculation of common components of a translation, usually the overlap of n-grams using some type of average.

Translation metrics is still a field of active development, but it's not uncommon for newer methods to be compared to the three metrics described here. In fact Google recently announced a newly developed metric they call “BLEURT” (Google AI Blog: Evaluating Natural Language Generation with BLEURT), so it's clear that BLEU is still often considered the metric to beat!

Supplement: Mathematical Details

Arithmetic mean: The NIST metric utilizes the arithmetic mean, which is the most common version of an average and is usually what is implied when someone uses the word “mean” on its own. The arithmetic mean is the sum of n numbers divided by n:

$$ A = \frac{1}{n} \sum_{i-1}^n x_i = \frac{x_1 + x_2 + ... + x_n}{n} $$

For instance, the arithmetic mean of 2, 4, and 3 would be $\frac{2 + 4 + 3}{3} = \frac{9}{3} = 3$

Geometric mean: The BLEU metric utilizes the geometric mean of the number of overlapping n-grams. The geometric mean is defined as the nth root of the product of n numbers:

$$ G = (\prod_{i=1}^n x_i)^\frac{1}{n} = \sqrt[n]{x_1 x_2 ... x_n} $$

For instance, the geometric mean of the numbers 1, 4, and 2 would be $(1 * 4 * 2)^\frac{1}{3} = \sqrt[3]{8} = 2$

Harmonic mean: The METEOR metric utilizes the harmonic mean of the number of overlapping n-grams. The harmonic mean is defined as the reciprocal of the arithmetic mean of the reciprocals of n numbers:

$$ H = (\frac{\sum_{i=1}^n x_i^{-1}}{n})^{-1} = \frac{n}{\sum_{i=1}^n \frac{1}{x_i}} = \frac{n}{\frac{1}{x_1} + \frac{1}{x_2} + ... + \frac{1}{x_n}} $$

For instance, the harmonic mean of 1, 4, and 4 would be $(\frac{\frac{1}{1} + \frac{1}{4} + \frac{1}{4}}{3})^{-1} = \frac{3}{\frac{1}{1} + \frac{1}{4} + \frac{1}{4}} = \frac{3}{1.5} = 2$

Supplement: Original Papers for Machine Translation Metrics

BLEU: BLEU: a Method for Automatic Evaluation of Machine Translation

NIST: Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics

METEOR: Meteor: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments

Read on A Marvelous Balance »

In any case it's now time for an upgrade. There's nothing obviously wrong with the site as it is, but there are some additional optimizations I could make and newer React patterns and best practices I'm keen on incorporating. First and foremost, I'm migrating the frontend framework I'm using from Create React App to Next.js. Next is also a React-based framework, but it incorporates several new features I haven't previously used such as server-side rendering and asset optimizations that should improve loading and rendering performance. Additionally, I'll be switching from standard JavaScript to TypeScript for better type safety, updating Material-UI to utilize server-side rendering, and incorporating modern React patterns like hooks and functional components. I have been attempting to perform this migration more directly, but after several cumulative hours of troubleshooting, reading documentation and Stackoverflow, and banging my head against the wall I've decided instead to wholly rebuild the frontend as a new Next app. Dealing with various build and undefined errors got frustrating and since this is a side-project I feel it should be a mostly enjoyable experience! Once the transition is complete the content should all be the same, but the final UI might look slightly different. And once that's completed (hopefully by New Year's!) I have a lot of interesting posts planned such as finally writing a walkthrough of the Urdu number classifier (note: this app is not currently available) and a post about the history of cryptography.

Stay tuned!

Read on A Marvelous Balance »

The short answer is that the exact size depends on the cryptographic algorithm being used, but the general best practice as of 2023 is to use a 2048-bit prime number for methods like the Diffie-Hellman key exchange algorithm. To give a visual representation of that, here is a (approximately) 2048-bit number derived using Python's cryptographic getPrime method:

I've noticed some recurring patterns in the ways people ask for help in these channels, and in this post I'll share some suggestions to both get the most help on your questions and also increase the chances that you'll solve the issue on your own. Going through these steps has the added benefit of improving your software development and debugging skills!

1. Share the text of the error message rather than a screenshot

A common type of help post I see is someone writing a brief message that lacks details on the problem they're experiencing, and the post commonly includes a screenshot of the error message and/or stacktrace¹. What is more helpful than a screenshot is a copy/paste of the text of the error message and stacktrace. This makes it easier for the OSS maintainers to read the error text and allows them to copy/paste the text to perform their own web searches, or search the OSS codebase for the error messages and names of functions in the stack trace.

2. Provide details on your environment

Along with including the text of error messages, it's often beneficial to describe the environment you're trying to run the OSS in. At a minimum, you should include the following information:

• The hardware (type of personal computer or server), operating system, and operating system version number
• The release or version number of the OSS you're using and seeking help with
• Details on any virtual environment, package manager, etc., being used (for example, maybe the person seeking help is using Anaconda for Python development, or running experiments within a Jupyter notebook) It can also be useful to describe what you're trying to do with the OSS, such as install it, use it within a larger program, etc.

3. Describe what you've tried so far

Lastly, you'll be more likely to get help if you can describe what you've tried so far to fix the issue. When interacting with senior colleagues in a professional setting, "What have you tried so far?" is often the first reponse you'll hear when you ask them for help to debug an issue. It can be useful if you describe what you've attempted to resolve the problem on your own, including:

• What have you searched for online via Google or other search engines
• Exchanges you've had with AI/LLM systems to debug the issue
• Attempts to rerun a command or process with different arguments, configurations, etc.

The great thing about detailing what you've tried is that it can frequently help you fix the problem on your own. This will especially be the case as you gain more experience. In my own career I can recall several times when I was stuck on an issue, and I began writing a message into Slack or Discord to ask for help. But in the process of writing the details of what the issue was and what I had tried so far to resolve it, I uncovered additional approaches to solving the problem that I hadn't tried yet. Frequently one of these new approaches would help me fix the issue!²

Helping OSS Devs and Yourself

To summarize, when you create a post asking OSS maintainers or the broader community for help, include the following information:

1. the text of the error message and stacktrace (rather than a screenshot)
2. the details of your environment and how you're using the OSS
3. a brief description of what you've tried so far to resolve the issue

Using the above tactics will help get you unstuck faster. You'll also be helping the OSS community more effectively find and fix bugs, and by using this process you'll be improving your own debugging skills to become a more effective software professional!

Read on A Marvelous Balance »

ZKPs are not a new idea, but they have experienced a recent growth in popularity, especially for applications in Web 3.0 and cryptocurrency domains. In this post I'll give a sense for how ZKPs work by walking through the process for one of the earliest examples of ZKPs, the Feige-Fiat-Shamir identification scheme. Note that the scheme relies on modular and exponential arithmetic, so familiarity with these concepts will be helpful.

Introducting the Cast

Similar to cryptocurrencies, many of the ideas in ZKPs originated in crypography. Discussions of cryptographic protocols usually involve "Alice" and "Bob" to represent two people sharing information, and another person, "Eve", attempting to eavesdrop on their communication.

For ZKP examples, instead of these three individuals we usually have the characters "Peggy," who is attempting to prove she knows a piece of information without revealing it directly (also known as the "Prover"), and "Victor", who is attempting to verify Peggy's claim (also known as the "Verifier").

The Feige-Fiat-Shamir Identification Scheme

This scheme was originally introduced¹ in 1988 by cryptographers Uriel Feige, Amos Fiat, and Adi Shamir. The essense of the scheme is that Peggy will prove to Victor that she knows some collection of numbers without sharing those numbers directly. Victor will challenge Peggy by picking random zeros and ones, and then a series of modular and exponential arithmetic steps are worked through that can only be solved with Peggy's secret numbers. This series of steps can be repeated any number of times until Victor is satisfied Peggy does know her numbers (and that she hasn't, for example, selected them at random).

Throughout this description we'll be using Python-based coding examples. The only extra dependency we'll need is SymPy for its useful isprime method.

Step 1: Generation of Random Primes and Peggy's Selection of Secret Values

Throughout this example, we'll use numbers between 0 and 1000, much smaller numbers than are used in practice², to make the process a bit easier to follow.

# initializing prime list
from sympy.ntheory import isprime

min_value = 1
max_value = 1000
small_primes = [i for i in range(min_value, max_value) if isprime(i)]

Now we select two random prime numbers from this list, p and q, and multiply them to create a third number n = p * q.

p, q = random.sample(small_primes, k=2)
n = p * q

Next, Peggy creates her secret numbers. These can be any amount of secret numbers, but for this example we'll randomly select three that we'll call s1, s2, and s3. Importantly, for the arithmetic to work out these numbers must be coprime to n and they therefore cannot equal either p or q:

s1, s2, s3 = random.sample([i for i in range(min_value, max_value) if i != p and i != q], k=3)

Lastly, Peggy also generates a random value r and computes a value x that "hides" the true value of r by squaring r and taking its modulous with n:

r = random.randint(min_value, max_value)
x = (r**2) % n

Step 2: Victor Challenges Peggy

For his part, Victor selects a random collection of 0's and 1's that we'll save as a1, a2, and a3 meant to challenge Peggy's knowledge of the secret s values:

a1, a2, a3 = random.choices([0, 1], k=3)

Victor then sends his challenge values to Peggy while she, in turn, sends him the value of x.

Peggy calculates a new value y that involves the product of the secret s values raised to the power of Victor's a challenge values, which is also multiplied by r and modulated by n.

y = (r * ((s1**a1) * (s2**a2) * (s3**a3)) ) % n

Peggy also computes values v that square the secret s values and modulate them with n:

v1 = (s1**2) % n
v2 = (s2**2) % n
v3 = (s3**2) % n

The v values as well as y are then shared with Victor.

Let's recap where we are at this point: Victor has a random selection of 0's and 1's that he shared with Peggy. Peggy has shared x, y, and the v and a values with Victor, but crucially she has not shared the secret values s nor the values n and r. In other words, Peggy has not shared the values she wants to prove she knows, but she has shared a number of values calculated (using modular and exponential arithmetic) with those secret numbers as well as Victor's challenge numbers, but all of the values Peggy has shared with Victor are numbers that were computed from n, r, and the secret values.

The next step is to prove that these computed values could only have been created if Peggy indeed knows the values of s.

Step 3: Peggy Proves her Knowledge of the Secret Values

This step involves one more calculation. Victor confirms that the value y shared by Peggy is equal to the product of x and the values of v raised to the power of his a values, all modulo n:

proof = (y**2 % n == (x * ( (v1**a1) * (v2**a2) * (v3**a3)) ) % n)
print(proof) # True

Mathematical Review

Let's review the definitions and equalities that support this proof. We'll go backwards through the steps we took above and use some mathematical notation rather than Python code this time:

1. By the definition of y our proof takes the form $$ mod(y^2, n) = mod((r^2 * ({s_1}^{a_1}) * ({s_2}^{a_2}) * ({s_3}^{a_3}) )^2, n) $$

From here we'll remove the mod operator for clarity.

2. By the definition of x we can replace the squared r term like so: $$ r^2 * (({s_1}^{a_1}) * ({s_2}^{a_2}) * ({s_3}^{a_3}) )^2 = x * (({s_1}^{a_1}) * ({s_2}^{a_2}) * ({s_3}^{a_3}))^2 $$

3. If we expand the squared terms to the right of x, we get $$ x * (({s_1}^{a_1}) * ({s_2}^{a_2}) * ({s_3}^{a_3}))^2 = x * ({s_1}^{a_1}) * ({s_1}^{a_1}) * ({s_2}^{a_2}) * ({s_2}^{a_2}) * ({s_3}^{a_3}) * ({s_3}^{a_3}) $$

4. Then applying the definition of the v terms, we get $$ x * ({s_1}^{a_1}) * ({s_1}^{a_1}) * ({s_2}^{a_2}) * ({s_2}^{a_2}) * ({s_3}^{a_3}) * ({s_3}^{a_3}) = x * (({v_1}^{a_1}) * ({v_2}^{a_2}) * ({v_3}^{a_3})) $$

Thus returning us to the proof statement (and with the mod operator re-applied): $$ mod(x * (({v_1}^{a_1}) * ({v_2}^{a_2}) * ({v_3}^{a_3})), n) = mod(y^2, n) $$

These steps confirm that Peggy indeed knows the secret s values and can prove it without needing to directly reveal those values.

There is a chance that Peggy could randomly generate a value for y such that the final step of the proof would be true without her actually knowing the secret values. And as mentioned before, the scheme designers propose doing several rounds of verification using different values for r and Victor's a challenge values where each successive and successful round would reduce the probability that Peggy is able to continually generate random values for y that also satisfy unknown values for the s secret values.

A Jumping Off Point for ZKPs

The Feige-Fiat-Shamir identification Scheme is a simple introduction to zero-knowledge proofs. The scheme is not used in practice due to its vulnerability to certain attacks, but it does demonstrate the core concepts of ZKPs such as proving knowledge of a value without revealing the value directly. More modern and robust ZKPs such as zk-SNARKs and zk-STARKs (types of non-interactive zero-knowledge proofs) are gaining adoption today, especially in the realm of cryptocurrencies.

If you're interested in learning more, I suggest checking out ZKProof, an organization committed to establishing global standards for ZKPs. They have a great list of resources to get you started. This Awesome Zero-Knowledge Proofs GitHub repository is also a great collection of links and resources worth exploring.

Read on A Marvelous Balance »

When deciding on a topic for my Springboard capstone project, I knew quickly that I wanted to work on a neural machine translation (NMT) model. Machine translation has been a focus of my studies within machine learning due to my facination with linguistics and learning foreign languages. I was drawn to work on a Japanese-English translator because of my familiarity with the language (I minored in Japanese in college and continue studying it in my free time) as well as curiosity to see how it would be to work with a language that does not use the Roman alphabet.

After several months of work I'm happy to share the progress I've made so far with this project. In this post I'll detail various aspects of the project from data gathering and cleaning to deployment, and I'll also give a walkthrough of the neural network architecture and ideas for future improvements to the project.

The Data

The dataset I chose to use for this project was the Japanese-English Subtitle Corpus, or JESC. This is a corpus of 2.8 million pairs of sentences taken from internet crawls of television and movie subtitles. Unlike most translation datasets, the JESC features a large amount of slang and colloquialisms which was part of its draw for me. It also did not require a large amount of processing to get it into a Python-friendly format as it came from a simple text file (as opposed to more complicated formats such as an XML document).

Preprocessing the data before feeding it to the neural network did require significant effort, however. This was largely due to some of the characteristics of Japanese text and how it gets encoded by computers. Let's go through a short description of written Japanese to get a feel for its differences from written English.

A brief overview of written Japanese

While English sentences are typically in the subject-verb-object order (“I go to the store”), Japanese is subject-object-verb (“I to the store go”). The written language uses 3 different alphabets:

1. Chinese characters called “kanji” (漢字) that denote ideas and thus are used to represent any words and can have multiple readings or pronunciations.

2. A native syllabary (meaning the characters represent whole syllables instead of individual sounds) “hiragana” (ひらがな), which is characterized by fewer lines per character than kanji and with more curved shapes. Hiragana is used for conjugation (often following kanji that form the first part of a word), grammatical parts-of-speech like adverbs and conjunctions, native words not represented by kanji, and several other purposes.

3. A second native syllabary “katakana” (カタカナ), characterized by few lines similar to hiragana, but with sharper lines and angles. Katakana is used for non-East Asian proper nouns and loan words, emphasizing text (like italic or bold letters in English), animal and plant names, and other uses.

Text Preprocessing

It turns out that because Japanese has multiple types of characters, native speakers can often use this feature to visually distinguish a sentence's individual words instead of using spaces as European languages use. This sentence, “I came from the supermarket”, features all three alphabets:

私はスーパーから来た。

While this visual division works fine for human readers of Japanese, it's still easier for software if it can break up English and Japanese text in similar ways, a process known in natural language processing as “tokenization”. Therefore we need a way to add spaces between logical portions of Japanese text, which can present challenges for things like compound words. For my project I used the popular NLP Python library Spacy along with the Japanese tokenizer library SudachiPy. This library actually has settings that allow you to vary the level of granularity in its tokenization. For instance, here is an example from the documentation showing the various options available to break up the word for “government official” (I opted for the middle-of-the-road option “B” in my project):

mode = tokenizer.Tokenizer.SplitMode.C
[m.surface() for m in tokenizer_obj.tokenize("国家公務員", mode)]
# -> ['国家公務員']

mode = tokenizer.Tokenizer.SplitMode.B
[m.surface() for m in tokenizer_obj.tokenize("国家公務員", mode)]
# -> ['国家', '公務員']

mode = tokenizer.Tokenizer.SplitMode.A
[m.surface() for m in tokenizer_obj.tokenize("国家公務員", mode)]
# -> ['国家', '公務', '員']

Another issue is that while Japanese has some obviously different puncuation marks (periods are “。”, for example), even punctuation that looks similar to the English counterparts (！or !) have different unicode values. Handling these as well as removing potential accented characters from the English text (i.e., turning “résumé” into “resume”) and transforming the text of both languages from Unicode into ASCII encodings was the majority of my pre-processing and text normalization work. Additionally, each sentence needs to start and end with special tokens (literally <start> and <end> in this case) as guidance for parsing each sentence.

Preparing The Text Data for Training

Since neural networks can only work with numbers, it's necessary to convert text data into numerical data for both the input language and target (or “output”) language of a NMT model. This process gives a numerical value to each unique word that appears in the data along with “word indexes” to be able to look up the words by their numerical values and vice-versa. The process also creates vectors with the same length as the longest sentence in the dataset (one for the input sentences and one for the target sentences). Each sentence is then one-hot encoded as a vector with numerical values at the space of each word and zeroes padding the vector to the maximum sentence length.

After this vectorization of the sentences has been performed, I split the data into the following ratios using Scikit-learn's train_test_split method: training data = 75%, validation data = 15%, test data = 10%. I then created a dataset using Tensorflow's tf.data.Dataset class.

Neural Network Architecture

The neural network used in this project was built using the Tensorflow framework, and the architecture I chose for this project was a sequence-to-sequence (seq2seq) model, more specifically an encoder-decoder model with attention. Seq2seq models use recurrent-neural-networks (RNNs), a natural choice for NLP problems as RNNs are capable of working with variable-length inputs and thus can handle text of various lengths. An additional appeal of RNNs, especially in NLP contexts, is that they have a sense of “memory” by being able to pass state between the layers of a neural network.

Depiction of a sequence-to-sequence model, "EOS" is "end of sentence" (Sutskever et al., 2014)

RNNs feature several sub-classes of networks such as long-short term memory networks (LSTMs). It turns out that standard RNNs have a difficult time if the space between references gets too long (an example would be having the context from the beginning of a long sentence to help translate the end of a sentence). This problem is referred to as the “long-term dependency problem”, and LSTMs were designed specifically to avoid this issue! They feature a concept called “cell state” that lets information pass between modules of a neural network to overcome the long-term dependency problem. My model specifically uses GRUs, or “Gated Recurrent Units” which, among other changes, simplifies the storing and passing around of this hidden state versus traditional LSTMs.

Visualization of an LSTM network cell (Source)

At the layer of the encoder-decoder part of the architecture, the encoder RNN reads in an input sentence and creates a vector representation of that sentence, which is then used by the decoder to extract a target sequence of that vector. During this process each word in the input sentence is assigned a weight value by the attention mechanism (described below), and this weight value is in turn used by the decoder to predict the next word in the target sentence.

Loss Function

This model used the sparse categorical cross-entropy method. Cross-entropy loss methods are also called “log-loss” methods and use a logarithmic function to increase the loss value as the predicted probability of an observation diverges from the actual observation. Sparse categorical cross-entropy is a specific use case for lots of categories (predicted translated words, in our case) that are represented as one-hot encodings, i.e. as vectors with a few ones and many zeros.

Attention Mechanism

The general idea of an attention mechanism is to prevent the model from associating a single vector with each sentence and use “attention weights” to influence which input vectors to prioritize and create a “context vector” that summarizes the input data received by the decoder. In encoder-decoder LSTMs without this attention mechanism, only the last hidden state value would be used to generate the context vector.

The network used in this project utilized Bahdanau attention, which works by using a linear combination of the encoder and decoder states to generate the context vector. The context vectors created by the Bahdanau attention mechanism as well as previously generated target words are then used by the model to predict a new target word. See the original paper that introduced Bahdanau attention, "Neural Machine Translation by Jointly Learning to Align and Translate", for more information.

Bahdanau Attention Diagram (Bahdanau et al., 2015)

Optimizer

The optimization algorithm used in this model is the Adam optimizer. This optimizer is an extention of the stochastic gradient descent algorithm, and improves on it by using an adaptive learning rate to speed up optimization. More technically, the Adam algorithm calculates the exponential moving average of the gradient and squared gradient and uses two parameters (beta1 and beta2) to control the decay rate of these two averages. The original paper that introduced the Adam optimizer can be found here: "Adam: A Method for Stochastic Optimization".

Model Performance

This model was trained in a Jupyter notebook with an accessible GPU hosted by Paperspace (the notebook can be found here: Encoder-Decoder model for Japanese-to-English Translation). After 100 epochs that ran for about 30 hours and another 12 hours for evaluating the model on test data, the process was complete.

I evaluated the models using the METEOR metric. Unfortunately, the performance was not as great as I had hoped (it rarely is!). The average METEOR score per translation turned out to be just 0.10576, however this doesn't mean that roughly 1 in 10 words matched the expected translation. Briefly, the METEOR metric calculates the harmonic mean of unigram recall and precision together while giving more weight to recall. See my post on evaluating machine translation models here for more details: "How to Evaluate A Machine Translation Model".

Deployment

Once training and evaluation were complete, I chose to deploy this model through this site (though as of 2023 I am no longer hosting it online). After saving the weights of the model using Tensorflow's own model saving method and saving the input and target language vectorizers as pickle objects, I built a class within a Django backend that instantiates a model, uses the saved, static weights and vectorizers from the Paperspace training, and returns a predicted English translation. The entire Django application runs within a Docker container.

Data flow in the deployed application

The application has basic error handling in place on both the frontend and backend, and logging on the backend as well.

Ideas for Further Development

Even though I was not satisfied with the model performance I remembered the maxim that “perfect is the enemy of done” and decided to continue with deploying the model, as completing an end-to-end machine learning project was more important than chasing a high metric score for too long. Nevertheless I'd like to share potential ideas for improving the model's performance and additional deployment ideas.

Data

Data is the fuel of machine learning models, and one of the first steps I intend to take when returning to this project is adding additional training data. While I thought it could be an interesting feature at the time, in retrospect having a dataset like the JESC that features a lot of slang hampers the model performance from a user standpoint, and it tends to be missing a lot of common vocabulary. Widely-used translation apps like Google Translate deal mostly with formal written language, and since translation apps have set a paradigm of avoid informal speech and text I would seek out additional datasets with more standard vocabulary. I would expect having additional training and testing data like this to have the largest impact on model performance.

Another idea is changes to the data preprocessing methods, especially choosing different options for how the Japanese and English text is tokenized as I described earlier. However, I would expect these changes to have less significant affects on the score versus changes to the data and model architecture.

Architecture and Hyperparameters

While using an encoder-decoder with LSTMs/GNUs are the dominant architecture choice for a machine translation model, some changes to the architecture are worth exploring. The initial change I would make would be to use standard LSTMs instead of GNUs within the encoder and decoder, as LSTMs have been shown to have better performance than GNUs for some neural machine translation cases.

Another change would be to the attention mechanism used. While I used the Bahdanau attention mechanism here I could also explore using Luong attention, which is a newer method that attempts to improve on Bahdanau attention by simplifying the calculations of attention scores, leading to a faster and more efficient attention mechanism. It also uses the decoder's current hidden state to create the context vector, unlike the Bahdanau's use of the previously computed hidden state.

A more significant change than these architecture adjustments I've discussed so far would be to use a model utilizing transfer learning with BERT, or “Bidirectional Encoder Representations from Transformers”. Transfer learning is a technique wherein neural networks trained for one task are used for predictions on a different (but usually related) task. Transformers are a class of neural networks with the encoder-decoder pattern that deal with long-term dependencies even better than LSTMs, and BERT is a class of transformer models that show excellent performance on a variety of NLP tasks than can be used with several pretrained models. Comparing my more “from scratch” model described in this post versus a transfer-learning-with-BERT model would be a very interesting follow up to this project.

Changing the hyperparameters could also have a significant impact on performance. However, because the size of the GPU memory limited how much data I could process at a time, the primary hyperparameter I could significantly adjust would be the total number of epochs. Nonetheless there's always the chance that training on the same data for longer could lead to overfitting, and given that training for 100 epochs took well over a day, fine-tuning this value could take a significant amount of time, assuming I'm limited to working with one GPU.

Deployment

The deployment is generally working fine, although one thing I would like to add is a caching layer to avoid recomputing estimated translations for common input values. For this I would probably use Redis or a similar in-memory database to store common input-output pairs of sentences.

If usage of the app grew large enough, considering strategies for deploying, managing, and networking multiple instances of the backend application would be fun. Using Kubernetes to orchestrate multiple Docker containers would be a natural choice here, and my existing NGINX configuration could be updated to handle routing requests to multiple backend servers.

Extra Lessons Learned

I greatly enjoyed working on this project, and I learned a great deal during the process. Often when I read other project walkthroughs I wish developers would share some “war stories” of unexpected hurdles, both to avoid giving readers the impression that development is very rarely a smooth process and to help readers avoid similar pitfalls. Accordingly, here are a two interesting issues I didn't expect to run into.

So much of the material I've read about machine learning and data science says that you'll spend much more time on data cleaning and preprocessing than you expect, and that was true for me as well. One particular error I dealt with that stood out was the appearance of two line separator unicode (U+2028) characters (which I had never encountered until now) hiding in 2 Japanese sentences, which didn't show up until I saved the processed data to local files. When I loaded these files the Japanese sentences would have 2 extra instances versus the English sentences because of these line separators, giving me an “off by 2” error amongst 2.8 million sentences!

Another tool's quirks that I noticed which will save me lost time from now on is some of the nuances of working with the Large File Storage Git extension, aka git-lfs or "LFS". LFS allows you to store large files on a site like Github using pointer files while the large file itself is stored on a different server. Frequently when I would clone my project from one environment to another, say from my local computer to Paperspace, I unconciously thought the large files themselves would be included in the cloning process, but I shouldn't have been surprised in retrospect that only the pointer file would move while I would have to manually copy/paste the large files themselves!

Acknowledgements

I want to give a huge thank you to my Springboard mentor, Dhiraj Kumar. During our weekly calls (which were late in the evening for his timezone!) he would regularly provide me with suggestions and feedback that kept me moving forward when I could have gotten stuck or spent too much time on a particular aspect or another of the project. Please check out his Medium blog and YouTube channel.

Read on A Marvelous Balance »

I've spent over 15 years working at a wide range of organizations, from early-stage startups to large government institutions. I've utilized multiple technologies building and maintaining many types of data-focused products. I've succeeded both as an individual contributor and team lead, and I'm comfortable working with and presenting to any section of an organization.

I believe Mitchell Technology Consulting¹ is the perfect way for me to work with a wide range of businesses to improve data utility and privacy in AI and other software applications. And I'm especially excited about the possibility of expanding the usage of privacy-enhancing technologies such as differential privacy – a method that allows us to analyze data without revealing sensitive individual information.

If you want to improve the way your team uses and protects your organization's data and the applications that use that data, schedule a free consultation today!

Read on A Marvelous Balance »

But I eventually acquiesced and once I got comfortable reading and writing Python I discovered some things I actually enjoyed about the language. In fact these were things I wish I could adopt into my JavaScript code. Below is a short list of these features:

1. Slicing Notation

This was probably the first part of Python's syntax that made me react with “Okay, that's a pretty nice feature.” Python's slicing syntax gives you the ability to easily get multiple subsections of any list (i.e., “array” in JavaScript). It looks like the following:

example_list = [1, 2, 3, 4]
example_list[:1] # -> [1]
example_list[1:] # -> [2, 3, 4]
example_list[1:3] # -> [2, 3]

As you can see, the slice notation syntax can be thought of as consisting of optional "start" and "stop" values like so [start:stop] (Technically both the "start" and "stop" values can be optional because using [:] will return a complete copy of the original list), but you can also use a "step" property to skip values within the sliced subsets like so:

example_list[0:4:2] # -> [1, 3]

2. List Comprehensions

List comprehensions are a prime example of one of the original goals of Python: to make code read more like a human language (at least if you're an English speaker!). Comprehensions give you a simple way to create new lists using this basic syntax:

[expression for value in iterable]

Here is a simple example:

doubled_list = [i * 2 for i in [1,2,3]]
doubled_list # -> [2, 4, 6]

You can even create nested comprehensions, which are especially useful when dealing with nested data structures such as matrices:

original_matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
transposed_matrix = [[row[i] for row in original_matrix] for i in range(3)]
transposed_matrix # [[1, 4, 7], [2, 5, 8], [3, 6, 9]]

Unfortunately, although list comprehensions were actually on the roadmap for ECMAScript 2015 and even implemented in some versions of Firefox, the feature was later removed: Array comprehensions | MDN (12/2021 update: comprehensions are being reconsidered for ECMAScript 7!).

3. Named Parameters

Python gives you the ability to both pass arguments to a method in a set order, just like you would in JavaScript, or create named parameters that are defined with a default value and that can then be passed in any order. Let's look at an example:

def rectangle_area(width=0, height=0):
    return width * height

I could call this method in any of the following ways:

rectangle_area(5, 7)
rectangle_area(width=5, height=7)
rectangle_area(height=7, width=5)

You can sort of adopt this pattern in JavaScript methods using objects:

function rectangleArea(rectangleObj) {
  return rectangleObj.width * rectangleObj.height
}
rectangleArea({ width: w, height: h })
rectangleArea({ height: h, width: w })

But I personally find the Python pattern easier to read at a glance, especially for methods with more than three or four arguments which are common in several data science and machine learning libraries.

If you're a developer who has worked with JavaScript but never with Python, hopefully this gives you a small taste of some of the things Python does well. Some features such as the mandatory whitespace and lack of keywords for variable declarations still look a little strange when I step back into Python after working in JavaScript for a while, but I am enjoying working with these features and many other methods in the standard library on a daily basis.

Read on A Marvelous Balance »

Before jumping into the content, I realize that sometimes a person can be in a desperate situation to find a new role and may be willing to take almost any offer that is extended to them. Trust me when I say I've been there: I've been laid off, I've struggled to escape a bad manager, and I've had successive rounds of bad interviews that killed my confidence. I was once even asked to leave an interview early because I was bombing the technical questions so badly. However, I think that even in those types of situations that it's nevertheless worth asking these kinds of questions because you'll at least have a better idea of what to expect. It's also worth asking many of these questions to multiple colleagues and managers, because if you receive wildly different answers it can indicate mismatched expectations or broken communication between managers and individual contributors.

I'll list out each question I like to ask by category, and then describe what type of insight the question should give you about the team/organization you're interviewing with.

Questions about Work Assignments

• Can you describe how work is typically assigned?

This should give you an indication of how collaborative the work environment is. If you're early in your career it may be expected to be assigned tasks without much input, but as you become more senior it can be demoralizing to have little or no say in your assignments. It will also prevent you from growing as an engineer from one who can complete a work ticket to one who can talk to users and customers to figure out their needs and begin architecting solutions in a team setting.

• How do you work with stakeholders and project managers to design or iterate on requirements?

This is similar to the above question, with more emphasis placed on iterative development and (hopefully) an answer will describe where you would fit within the development of features and products.

Questions about Fires 🔥

• Can you tell me about the last team emergency you had, like a very unhappy customer or broken deployment, and how the team overcame it?

It's all fine to learn how a team works on a normal day, but what about when things go really wrong? These are the types of days you'll most strongly remember, for better or for worse. How managers and teammates respond on these types of days can instill loyalty and comraderie for strong teams, or bitterness and resentment for weak ones.

• How does the team learn from mistakes like these and institute changes so that mistakes aren't repeated?

It's possible that in a young organization (like an early-stage startup) that processes such as post-mortems have not yet been instituted, or that no emergencies have been encountered. In that case, I typically like to create a hypothetical scenario such as in the previous question and ask how they might respond.

Skills and Career Development

• How does the team support career development (conferences, continuing education, etc.)?

Similar to the fires question above, early-stage startups rarely have the resources to support much career development. But hopefully in lieu of money for conferences or classes they can provide time for hackathons or try to provide formal or informal mentoring. Leaders who haven't thought about this kind of thing at all may present a yellow or red flag as an organization that will burn you out with constant work with few new skills or certifications to show for it when you start searching for your next opportunity.

• How do your performance reviews work?

This question should give you a clear insight into how the organization thinks about career development and promotions, and how you'll be expected to report on your work.

Role Expectations

• If a candidate were to accept this offer and it was 6-12 months from now, what would you imagine a wildly successful candidate to have done in that time?

This is an excellent question to end on. It can create a sense of excitement for both parties and often give you insight into what some unwritten expectations are and what the organization deeply values in an employee. If you get an offer and can execute on the answer to this question, you'll likely be on the fast track to recognitions of your hard work such as bonuses and promotions.

This isn't an exhaustive list, and obviously there may be questions you can ask that are more specific to the role or organization. But this should give you some ideas about how to get the most out of your Q&A time as a candidate, and hopefully at the end of the process you'll have a clearer picture of the team that you are considering joining.

Best of luck!

Read on A Marvelous Balance »

As an example, say we're creating a derived column and then we want to group the results by that new column. In this code example we'll create a new column summing the value of multiple customer contracts from a hypothetical database:

SELECT
    customer_id,
    contract_1,
    contract_2,
    contract_3,
    SUM(contract_1, contract_2, contract_3) AS contract_totals
  FROM contract_table;

In the case where you want to order by the size of the contract totals, you would likely encounter an error stating that the column contract_totals does not exist:

SELECT
    customer_id,
    contract_1,
    contract_2,
    contract_3,
    SUM(contract_1, contract_2, contract_3) AS contract_totals
  FROM contract_table
  ORDER BY contract_totals;

It turns out that standard SQL runtimes will attempt to run a “group by” statement before the “select” statement, and in fact the “select” statement is one of the last parts of a SQL query that gets executed. The following is the order that a typical SQL statement is executed in with a brief description of that command:

1. FROM: pick the table(s) to be queried
2. WHERE: filter the rows
3. GROUP BY: aggregate the rows
4. HAVING: filter the aggregated rows
5. SELECT: select the columns that appear in the output
6. ORDER BY: sort rows by value(s)
7. LIMIT: restrict the maximum number of returned rows

Unfortunately there's not a great way to work around this process and your best bet is to have this order of execution memorized. For instance, returning to our previous example with the contract amount summation, it's easiest to repeat the sum expression within the ORDER BY statement:

SELECT
    customer_id,
    contract_1,
    contract_2,
    contract_3,
    SUM(contract_1, contract_2, contract_3) AS contract_totals
  FROM contract_table
  ORDER BY sum(contract_1, contract_2, contract_3);

Even though this should set off every programmer's DRY alarm (DRY=“don't repeat yourself”), writing your SQL queries with the execution order in mind will reduce the number of errors you encounter, especially as you transition from a SQL newbie to a seasoned user!

Read on A Marvelous Balance »

Last week I successfully completed the Certified Information Privacy Technologist (CIPT) exam offered by the IAPP, the International Association of Privacy Professionals. While preparing for different privacy engineering¹ interviews over the last several months (several of which listed the CIPT as a desired certification for candidates), I realized I had some gaps in my knowledge when it came to working with teams that are not necessarily software engineers, such as legal and compliance professionals. Studying for this exam did end up closing those gaps and I can see some new paths forward in my privacy career as a result of getting this certification. If you think getting a privacy engineering certification could be beneficial for your career², read on for my perspective on how to study for and pass this exam.

What are the IAPP and CIPT?

The IAPP is a professional network headquartered in the US state of New Hampshire and with active local chapters around the world. The organization supports different means of educating and certifying those professionals in all aspects of privacy: legal regulation, corporate policy guidance, technical implementation, and many more. They offer several different certifications focused on implementing and maintaining technical privacy controls (as well as a newer certificate specifically for AI governance).

The CIPT itself is an accrediation that demonstrates an individual can implement technical privacy through various means, such as:

• Designing, building, and maintaining systems that protect user privacy
• Establishing best practices for data privacy and security
• Performing audits and working with teams across an organization to implement privacy solutions

Overview of the CIPT Exam

At first glance the exam doesn't seem too onerous: it consists of 90 multiple choice questions that you have two and a half hours to complete. But because it covers such a wide range of topics it has a reputation for being somewhat difficult, especially if you don't commit to focused study. While it does include directly technical topics such as encryption and computer networking, it also covers several project planning and organizational topics, such as designing software with a privacy by design first approach and how to implement a privacy policy within an organization. More philosophical and legal topics are also covered, such as a comparison of various definitions of privacy itself, and what actions constitute a violation of privacy.

The full list of sections covered is:

1. Foundational Principles
2. The Privacy Technologist's Role in the Context of the Organization
3. Privacy Risks, Threats and Violations
4. Privacy-Enhancing Strategies, Techniques and Technologies
5. Privacy by Design
6. Privacy Engineering
7. Evolving or Emerging Technologies in Privacy

Costs

At $550, the cost of the exam is steep (though this is comparable to other technical certifications such as those offered by CompTIA), and it includes a 2-year maintenance fee of $250 (waived if you're already a paying IAPP member) and continuing education requirements. So I wouldn't consider taking this exam if you're not working in a privacy-related domain.

Devising a Study Plan

Learning Materials

The IAPP offers its own study textbook, which is a good start, and they also offer reading material and practice tests that you can purchase. But after doing some research I decided to go with using the Privacy Bootcamp site to study for the exam. This is an online platform that features detailed content on all of the relevant CIPT topics, much of it pulled directly from IAPP material. The platform also includes review notes and flashcards and practice tests as well. Similar in price to the exam itself, Privacy Bootcamp charges $499 for access to its CIPT content, though that does include a guarantee for a full refund if you fail the first attempt at the exam.

Using a Spaced-Repetition System

For my own exam prep, I typically spent between 15 and 60 minutes a day reading and reviewing the Privacy Bootcamp content and then extra time here and there making and studying flash cards using Anki. If you're not familiar with Anki, it's a flash card-based spaced repetition system (SRS). The basic idea of an SRS is that each time you successfully remember review material, more and more days will pass before you see that material again, and any questions you get wrong will reappear more immediately (i.e. a few minutes later) until you get them correct. It's common for Anki users to make their card decks publicly available but I strongly encourage you to create your own cards, as figuring out what content to use to create a new card is itself a great form of study.

If you want to learn more about strategies for how to create cards to effectively absorb material, I strongly recommend this blog post on using SRS to study mathematics (although the strategies translate to other technical topics too): https://cognitivemedium.com/srs-mathematics.

Setting a Schedule

Each day I would make a commitment to read one to three sections of the Privacy Bootcamp material and then study the in-platform flash cards and knowledge review questions. I would also use the knowledge review material to make my own Anki cards, which I would then periodically study throughout the day.

Reviewing 50-100 Anki cards everyday typically takes me 5-20 minutes, depending on how unfamiliar the card content is. And it's an easy task to fit in between calls or during other small breaks during the day.

Practice Exams

After completing all the lessons, I took the two practice tests that are included in the Privacy Boot Camp platform. After taking the first one and passing it, I scheduled my official CIPT exam for the following week.

All of the questions in the exams fit the same format: reading content (sometimes with a longer "privacy scenario" that applied across multiple questions) along with four multiple choice questions. Part of what makes the test challenging is that very often you will get a question where two of the possible answers are both strong contenders. The challenge is to pick which one most strongly applies to answering the question.

Next Steps

The IAPP runs knowledge-sharing groups (which they call "peer groups") for disciplines like privacy engineering, and they frequently hold talks and conferences virtually and in-person around the globe. I've attended talks on subjects such as technical implementation strategies for data protection and the application of laws in the US/UK/EU on automated decision making systems.

if this overview has piqued your interest enough to consider taking the exam, explore a few of the (non-affiliate) links I've placed here. And best of luck!

Links

• CIPT Body of Knowledge and Exam Blueprint

• An Introduction to Privacy for Technology Professionals Second Edition

• Privacy Bootcamp

• Anki

• IAPP Privacy Engineering Peer Group

Read on A Marvelous Balance »

I hit a few roadblocks in trying to incrementally update my existing website's frontend codebase from a standard React- and TypeScript-based app to a modern Next.js-based application, and more than once I stepped away from this work for a few months. Probably the most difficult challenges included trying to update several Material UI components between major versions of that library, including some API changes that made the site's responsive design break in sometimes subtle ways.

Ultimately I decided to rebuild the site using a blog template I found on the Next.js site. The template includes integrations with the Tailwind CSS framework and some easy configurations for email and site analytics. I also appreciate that the template has built-in support for both dark and light themes, and the usage of the great MDX tool makes writing new posts much easier. On the previous version of the site I was manually encoding the HTML and JSX around the text of each post! Additionally, Vercel and Next.js apps include great performance when it comes to things like page loading and accessibility, as verified by high scores using Google's Lighthouse developer tool, as well as an A+ SSL report from Qualys.

And speaking of making updates easier, I've migrated the cloud hosting platform for this site from DigitalOcean to Vercel (probably not too big of a surprise since Next.js is a Vercel-supported framework). In order to make updates to the previous version of the site, I would have to SSH into my DigitalOcean server, do a git pull on the website repository, and then rebuild new Docker images before launching the containers (I also used Docker's image registry to skip the git pull and image build steps). Using Vercel's serverless architecture and support in Github Actions makes writing a new post much easier; essentially I just need to create a new markdown file (and upload any associated images), and then push these to the deployment branch on Github. Greatly reducing the friction to updating the site will it make that much easier to write new content. One cost of this migration is the Django backend I hosted on DigitalOcean that allowed me to support APIs for my Japanese-English translation and Urdu numeral-recognition systems, but I may try to find a way to restore support for these in the future. As is, they were used very little and their large sizes meant I was paying more than $20 per month in cloud usage fees.

I don't mean for this to sound like a knock on DigitalOcean. On the contrary, it is a great platform that helped me learn some dev ops basics such as setting up and maintaining a Linux box for deploying a web application, as well as some more complicated processes such as configuring NGINX, Let's Encrypt, and other services to enable a modern web application. They also have some of the best online documentation and tutorials for setting up and maintaining cloud applications and dependencies. They'll continue to be one of my top choices for cloud providers in future projects. But it is hard to beat the convenience of a serverless architecture for a personal blog, and especially when that includes Vercel's integrations and support for a Next.js codebase hosted on Github that has a generous free tier for personal projects.

Lastly, I also took some time with this refactor to do a little branding. The golden ratio (φ) is one of my favorite mathematical constants, so I decided to make an SVG of the golden rectangle (a rectangle with side length proportions are equal to the golden ratio) based on an image on Wikimedia that's in the public domain. Relatedly, I thought to give the website a name rather than just “Curt-Mitch.com.” Playing off of the golden ratio idea, I thought “A Marvelous Balance” seemed like a good name. It feels like an elegant title for a website that contains writings on multiple interrelated topics.

In any case, I'm looking forward to writing interesting content about a variety of topics on a more consistent cadence. If you'd like to, check out the repository for this site here: https://github.com/curt-mitch/curt-mitch.com/.

Thank you for reading!

Read on A Marvelous Balance »