2020/08/08

Gitlab + letsencrypt + Private network

Requirements

This blog requires you to have following items:
  • Gitlab server
  • TLS from let’s encrypt
  • Private network, which outside world cannot reach directly
  • You own a public domain name
If your gitlab server is reachable from outside world, you can reference manual here for setup let’s encrypt.
For poor souls like me, stay tune, and keep reading.

What is let’s encrypt

For detail, please read their website yourself. To me, it is a service for me to have a valid TLS certificate but free of charge.

Gitlab server + letsencrypt

According to my experiments, till Gitlab 12.9.2, Gitlab does not support requesting certificate via DNS challenge. So, you can just disable the let’s encrypt configuration provided by gitlab if your gitlab server is not public at all.
Surprisingly, we can still benefits from let’s encrypt since it will still offer you a TLS certificate, and private key, even though you need to do the verification every 3 months.
In other words, once you have certificate, and private key, you should configure your gitlab to read them as if you are using traditional TLS certificate, which bought from some vendors.

Request certificate from letsencrypt via certbot

Firstly, you should get a certbot by following let’s encrypt manual here in your gitlab server host.
Then, you can run following command to request a certificate from letsencrypt with DNS challenge for verifications.
certbot -d YOUR_GITLAB_SERVER_DOMAIN_NAME --manual --preferred-challenges dns certonly
Follow instructions there, you will finally have a DNS challenge string. Setup a TXT record with that DNS challenge string in your public DNS.
Below is a picture I capture from Google to illustrate the txt record setup.
Once you have complete the DNS challenge, you should get a certificate, and private key as stated in the command output. Personally, I suggest you to write down those important notes generated into a file for future reference.

Configure gitlab server to read certificate from gitlab server

By default, you should have following default configurations in /etc/gitlab/gitlab.rb
# nginx['ssl_certificate'] = "/etc/gitlab/ssl/#{node['fqdn']}.crt"
# nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/#{node['fqdn']}.key"
You can overwrite them with value you like. For me, I just stick to the default configuration. Therefore, I copy the certificate, and private key to the destination above.
It is worth to point out that you should name your certificate as YOUR_DOMAIN_NAME.crt, and private key as YOUR_DOMAIN_NAME.key if your are using default configuration like mine.
Finally, you can restart your gitlab server, and checkout your new certificate by reading them in your browser.

Probability speaking: How hard to get 20 wins on No tilt in Clash Royale

Recently, I have read a post from a forum which calculate the probability of getting 20 wins in No tilt in Clash Royale. IMO, this is a clever, and objective way to tell others that how hard, and how valuable to get 20 wins from No tilt!
For example, even you are top players with 80% win rate, you still only get 15.45% chance for having 20 wins. For modest players with 60% win rate, you only have 0.16% for having 20 wins!
Since the author does not expose the formula for the calculation, I would like to find it out here due to curiosity.
For Chinese readers, you can take a look the spreadsheet I wrote here

Facts

Before the calculations, we need to know following facts:
  • At most 20 wins
  • In any time, you can have at most 3 loses
  • You always start from 0 wins

Calculation

I strongly recommend you to cross check with the spreadsheet while reading below steps.

Step 0: Denote symbols

For brevity, we denote following symbols
W: Win rate. Chance to win a match
L: Lost rate. Chance to lose a tournament (IE 3 loses)
X: Number of wins in a tournament
N: Number of matches

Step 1: Define the win rate

To determine the probability for you to get 20 wins, we need the win rate of yourself. Generally, you can pick this up from your player’s statistic.

Step 2: Calculate the lose rate

L = (1-W)(1-W)(1-W)
This value is essential for us to calculate following 3 interested probabilities:
  • Probability of only X wins
  • Probability of at most X wins
  • Probability of at least X wins

Step 3: Probability of only X wins

Generally, we just need to pay attentions on 3 scenarios before we can deduce a formula to fit P(only x wins)
  • P(0 win)
  • P(Only 1 win)
  • P(Only 2 wins)

Step 3.1: P(0 win)

Probability for 0 wins equals to the Lose rate. IE, you lose 3 consecutive times since the tournament start.
P(0wins) = L

Step 3.2: P(Only 1 win)

You get 1 win, and lose 3 times.
P(wlll) = LW
Recall the combination theory you learn from secondary school, you can also get following combinations for achieving 1 win 3 loses.
  • wlll
  • lwll
  • llwl
  • lllw
The last one (lllw), however, is an invalid combination due to facts above. So, we need to cross out it. Finally, the result will be like this:
P(1w3l) = p(wlll) + p(lwll) + p(llwl) = WL * 3

Step 3.3: P(Only 2 wins)

  • Firstly, we need to calculate the probability of wwlll first.
P(wwlll) = LWW
  • Then, we just need to sum up valid combinations of 2 wins 3 loses in order to get the result we need.
P(wwlll) = LWW * Z, where Z is number of valid combinations.
The question is how to determine Z? Luckily, we can rely on Combinations Calculator (nCr) to do the mathematics for us.
The combinations of 5 matches with 3 loses = 5C3 = 10
  • Thirdly, we need to minus invalid combinations (LLL, LWLL, LLWL, LLLW) from above combinations:
Number of valid combinations = 5C3 - 4 = 5C3 - 4C3
  • Finally, we have following results
P(2W3L) = WWLLL (5C3 - 4C3) = WWLLL 6

Step 3.4: P(Only X wins)

It is obvious that there are 2 factors in the formula:
  • First part is the x win in N
  • Second part is how many of valid combinations for x win in N
The first part can be deduced as L(W)^x.
The second part can be deduced as (N)C3 - (N-1)C3 since number of invalid combinations for N matches equals to the valid combination for (N - 1) matches.
Finally, we have
P(Only x win) = L(W)^x * ((x+3)C3 - (x+3-1)C3)

Step 4: P(at most X wins)

Since we know P(only x wins), we can calculate probability of at most X wins by following:
P(at most 0 win) = P(0 win)
P(at most x win) = P(only x win) + P(at most (x-1) win), for x ranges from 1 to 20

Step 5: P(at least X wins)

Since we know P(at most x win), and P(only x win), we can calculate P(at least X wins) as following:
P(at least 0 win) = 1
P(at least x win) = P(at least (x-1) wins) - P(only (x-1) win)
Congratulation. That’s the end of the calculation.

At last

If you can read up to here, thanks for the reading. I hope you enjoy the path for solving the probabilities by ourselves especially that no ones want to explain just like the comments in the original forum post.

2020/08/01

Use python's pathlib to implement file path across OS

Why file path is an issue across platform

Nowadays, there are 3 common OS, which are Windows, Linux, and MacOS. Since Linux, and MacOS shares same file path’s implementation, developers do not need to worry about file path issues on between them. The problem is between Windows, and others.
In Windows, file path can be expressed in either 2 formats:
  • \folder1\folder2\file1.txt (Window format)
  • /folder1/folder2/file1.txt (POSIX format)
The difference is the direction of the slash. A side note is that file path in Linux / MacOS is expressed in POSIX format ONLY.
Actually, if applications in Windows can eat them both properly, I don’t need to write this blog since I can simply written in 2nd format at all. According to my experiences, even applications in Windows accepts either one of them only. It is a try and error process for picking a correct format for each applications.
Back to the title of this blog, file path issue will be bigger if your implementation runs across platform. For example, a library (Python zipfile library) may accepts POSIX format but the format of input file path may be Window format at all. So, the question is whether python provide library for us to do this conversion or not.
A side note on python zipfile. This library accepts both kind of formats since py3.8

pathlib

Pathlib is a builtin library for us to fix this problem. Below demonstrates 2 ways on conversion via such library. For other advance usage, please consult the library here.

Get POSIX file path from different input format

import pathlib
winPath = r'\workspace\xxx\test_fixture\user-restore-success.zip'
posixPath = '/workspace/xxx/test_fixture/user-restore-success.zip'
pWIN = pathlib.PureWindowsPath(winPath)
pPOSIX = pathlib.PureWindowsPath(posixPath)
pWIN.as_posix()
#'/workspace/xxx/test_fixture/user-restore-success.zip'
pPOSIX.as_posix()
#'/workspace/xxx/test_fixture/user-restore-success.zip'

Get Window file path

str(pWIN)
str(pPOSIX)

Notes

  • Always favor PureWindowsPath when doing conversion
PureWindowsPath is able to convert between Window path, and POSIX path. PurePosixPath, on the other hand, is not able to do so since Backslash () is a valid filename in POSIX path.
For detail, please refer to the discussions here
  • Bug in Path.resolve() on Windows platform
According to here, Path.resolve() in windows cannot return the absolute file path if such file is not existed at first. If you need a reliable way to get absolute path of a file right now, use os.path.abspath instead