Setting up a “slim edge” with kernel packet forwarding and encryption

I have some VPS servers with good connections and rather limited hardware resources, which cannot be used as an edge node to handle Kubernetes traffic by itself. For instance, a VPS server with CN2-GIA connection is perfect for Mainland Chinese visitors, but it comes only with a shy 512 MB of memory, which is a little tight for running Docker, Kubelet and Nginx Ingress Controller.

So I am trying to set up an forwarding mechanism to proxy the traffic between visitors and another more powerful Kubernetes node. A simple solution might be to run a TCP port proxy, such as nginx, haproxy, socat, or even iptables’ MASQUERADE feature from the Linux kernel. The problems with this method are:

  • The original visitor IP will be lost and replaced with the edge’s IP.
    • Surely you can use the PROXY Protocol to wrap and forward the client IPs, but that would require additional setup. For instance, Nginx Ingress Controller allows you accept only traffic with or without PROXY Protocol, but not both.
  • With the exception of kernel forwarding, traffic must be handled by the user space, which can increase the load on an underpowered node.
  • Traffic is forwarded between nodes as-is, with no additional encryption.
    • This will be problematic if the two nodes are not connected with a private network, and the traffic forwarded is in a plaintext protocol.

I experimented with a new solution, which overcomes all the problems mentioned above. Here is an overall illustration of the setup:

Traffic flow between clients and servers
Traffic flow between clients and servers

The following are the incoming and outgoing traffic flows, along with the commands used to set up forwarding:

  1. Incoming traffic
    1. When a client from [Client.IP] hits [Slim.Edge.Public.IP]:80, the traffic will be received by the edge server’s public network interface.
    2. Edge server changes the destination address to [Powerful.Node.Private.IP]:80 using the kernel’s DNAT mechanism. This happens in the pre-routing stage.
      iptables -t nat -A PREROUTING -p tcp -m multiport --dports 80,443 -d [SLIM.EDGE.PUBLIC.IP] -j DNAT --to-destination [POWERFUL.NODE.PRIVATE.IP]

      • Note that unlike other proxy setups, there is no masquerading (iptables’ MASQUERADE action) in place, because we want to preserve the original source IP.
    3. The kernel will pass on the incoming traffic to edge server’s WireGuard interface. By default, forwarding traffic is not allowed, and a forwarding rule has to be added explicitly.
      iptables -t filter -A FORWARD -o [SLIM.EDGE.WIREGUARD.INTERFACE] -j ACCEPT

      • Here [SLIM.EDGE.WIREGUARD.INTERFACE] is the slim node’s WireGuard interface, such as wg0.
      • Also remember to enable the net.ipv4.ip_forward toggle. This is pretty routing, so I won’t get into details.
    4. Powerful node will receive the traffic on their WireGuard interface, and since the ingress controller (or HTTP server, or whatever daemon you are trying to forward the traffic to) is listening on the host interface with a wildcard address (0.0.0.0), the forwarded packets will be handled just like direct ones, with the client’s source IP addresses in tact.
  2. Outgoing traffic
    1. The powerful node will probably reply with a packet from [Powerful.Node.Private.IP]:80 to [Client.IP]:(source-port).
    2. Use an IP rule to make sure that traffic coming out of the private interface, regardless of the destination address, will be sent to the WireGuard interface.
      ip rule add from [POWERFUL.NODE.PRIVATE.IP] table 1234 prio 5678

      • Note that the routing table ID (1234) should be set in the WireGuard configuration (Table = 1234) in order for the wg-quick script to create and fill this routing table. The priority number (5678) can be anywhere between 1 and the main rule (usually 32766).
      • When setting up the WireGuard tunnel, make sure to put AllowedIPs = 0.0.0.0/0 (or the IPv6 counterpart ::/0) in the edge node’s peer definition.
      • You can examine the generated routing table by running: ip route show table 1234, and the output should be similar to: default dev [POWERFUL.NODE.WIREGUARD.INTERFACE] scope link
    3. The traffic will be received by the edge node, which it needs to change the source IP from [Powerful.Node.Private.IP] back to its own public IP, using the kernel’s SNAT mechanism.
      iptables -t nat -A POSTROUTING -p tcp -s [POWERFUL.NODE.PRIVATE.IP] -j SNAT --to-source [SLIM.EDGE.PUBLIC.IP]
    4. From the edge node’s perspective, this packet is also a forwarded one, so another forwarding rule should be added.
      iptables -t filter -A FORWARD -i wg-pe-ba-la -j ACCEPT

With this setup, I can achieve the goal of receiving traffic using the slim edge node, while avoiding the issues mentioned above: The client IP is preserved during the whole process, the traffic forwarding happens entirely in the kernel space, and traffic between servers are encrypted.

However, the encryption is totally optional, so the WireGuard tunnel between servers can be easily replaced with other layer-3 tunnels like IPIP or GRE.

小技巧:Windows 系统下 Alt+F7 快捷键失效?

最近发现在我的 Windows 系统下,JetBrain 全家桶(IDEA、PyCharm、PhpStorm 等等一系列 IDE)的 Find Usages 快捷键 Alt+F7 不响应了。对于使用 IDEA 默认键盘配置的用户来说,“查找引用”的使用频率还是挺高的。

本来是想使用一些检测快捷键冲突的软件查找一下,后来简单 Google 了一下,发现很多人都遇到了这个问题

罪魁祸首是 N 卡显卡驱动自带的 NVIDIA GeForce Experience 软件,它有一个“游戏内覆盖”的功能,主要触发快捷键是 Alt+Z,但是会把其它一系列快捷键 Alt+F1, Alt+F2, Alt+F3, Alt+F5, Alt+F6, Alt+F7, Alt+F8, Alt+F9, Alt+F10, Alt+F11 都注册掉。(思考题:为什么没有 Alt+F4?)

解决方法就是在开始菜单中启动 GeForce Experience 软件,点击右上角齿轮进入设置,直接禁用“游戏内覆盖”功能。如果确实用到游戏内覆盖界面,也可以按 Alt+Z 打开覆盖界面,在设置中的“键盘快捷键”中修改或删除冲突的快捷键。

xiaodu-jsdelivr: WordPress Plugin To Scan and Serve Static Files From jsDelivr CDN

I created a plugin called “xiaodu-jsdelivr”, which automatically scan and replace references to WordPress static files, including plugins and themes that can be found in the official repository, with their canonical jsDelivr CDN addresses.

The plugin has been uploaded to the official WordPress plugin category. You can search for “xiaodu-jsdelivr” in the plugin installer, or download the ZIP archive here to install.

How it works

I downloaded and tested some of the existing WordPress plugins with jsDelivr replacing feature, and noticed that most of them employ a passive approach, which is to wait for visitors to request some static files, and then search the file hash against jsDelivr’s lookup API.

My plugin instead uses a more proactive approach. It starts by scanning static files directly from the WordPress installation directory, instead of waiting for visitors’ requests. To do this, I used the official WP-Cron scheduler to perform scans in the background at fixed intervals.

Then it will calculate local file hashes and compare them with known URL patterns (both WordPress itself and its official plugins and themes are supported), and matched URLs will be recorded and later used to replace local file references when they are requested. This is more reliable than just using the lookup API, because it can match against files that have never been visited by anyone, thus cannot be discovered simply by looking up hashes.

Here is a screenshot of what wp-content references can transform into with the plugin enabled:

Demo of plugin

Note the four different kinds of successful scan results shown in the screenshot above: Base WordPress files (line #32 – #35), an official plugin (#36), an official theme (#38 and #42) and custom files that the fallback hash lookup successfully found (#37).

Why jsDelivr

There are a handful of popular and reliable static file CDNs available. I think other plugin developers and I probably chose jsDelivr for the same reasons:

  • It has native WordPress support, namely https://cdn.jsdelivr.net/wp/plugins/ and https://cdn.jsdelivr.net/wp/themes/ that point to official plugin and theme SVN repositories. Other static CDNs mostly just load from GitHub and/or NPM.
  • It is more reliable in Mainland China, thanks to the QUANTIL (ChinaNetCenter / WangSu) nodes it uses for the Chinese visitors. Their overall visitor performance is better than their alternatives. The alternatives usually perform pretty bad in PRC.

Future development

The latest stable version will always be published to the WordPress plugin category. The plugin is GPLv2 (or later) licensed, and the first version (1.0) comes with full functionality and all the features mentioned above.

Future development will be carried out in the public GitHub repository.

小技巧:联通 PPPoE 拨号失败问题

作为多年的联通宽带用户,发现联通 PPPoE 拨号有一个问题,就是如果在短时间内多次拨号(例如频繁插拔路由器,或电脑多次断开重连),就会出现拨号失败、没有响应的情况(电脑端可能会有 678 错误)。

此时上游线路(光纤)都是没问题的,账号、密码也没有错误,但是一直无法拨号成功。

这种情况下,可以尝试这种方法:暂停拨号一段时间,比如将路由器断电、或电脑关闭拨号界面,等过 5 分钟以上再重新拨号,应该就能成功。

如果尝试之后依然失败,或上游线路本身就有问题(可以在光猫管理界面中看到光信号状态),则应该拨打 10010 报修。

LastPass “难民” 的新选择:KeePass

随着 LastPass 对免费用户设定的最新限制,包括我在内的一大批白嫖用户面临着移动端或 PC 端无法访问的困难。于是最近终于下定决心寻找 LastPass 的替代品,最好要是开源软件,并且能自己管理、同步数据。这样的选项并不多,最终我选择了 KeePass

之前 LastPass 为免费用户提供的功能主要有:

  • 密码跨设备同步
  • 浏览器端密码自动填充,自动提示添加
  • 手机端密码自动填充、自动提示添加

经过一番查找,同时满足开源、自己管理数据两个条件的成熟密码管理软件共有两个:KeePass 和 BitWarden

对 BitWarden 的考虑

BitWarden Logo

首先值得肯定的是,BitWarden 作为商业化运营的软件,官方提供的软件集成度和丰富性相对 KeePass 都要高很多;因为后者除了官方桌面客户端以外,其它功能(包括同步、移动客户端、浏览器插件)都是靠第三方插件和软件实现的。

BitWarden 的开源程度也比较高,提供了客户端的源代码和自己托管服务端的选项。但我也发现了一些问题:

  • 自己托管服务端需要启动 10+ 个 Docker 容器,其中包含独立的 Microsoft SQL Server,因而最低配置要求也较高,内存需要 2GB 以上。
    • 他们应该是直接提供了官方服务端,实际上个人托管的话,一个 SQLite 数据库也就足够了。
    • 更新(2021.4.12):GitHub 上有第三方提供的轻量级服务端,可以兼容官方客户端 API,比较适合自行部署。
  • 自己的服务端环境依然需要依赖 BitWarden 提供的中心服务器。
  • 部分付费功能,即使自行托管服务端,依然需要购买付费套餐,通过验证后才能解锁

因此,最终还是没有选择 BitWarden 的方案。对于不喜欢折腾的用户,可能直接购买一个 BitWarden 的付费套餐会比较划算,毕竟 $10/年的价格相对其它竞品很有优势。

选择 KeePass 方案

最终决定迁移到 KeePass,经过一番对比,采取了以下软件组合:

  • 桌面客户端:KeePass 官方版
    • 功能完善,支持各种格式导入/导出、自动填充、多语言等
    • 通过插件扩展机制,可以支持更多同步方式、第三方集成等功能
    • 使用 .Net 编写,通过 Mono 可以支持 macOS 和 Linux 下运行
  • 浏览器端:Kee 插件
    • 开源,功能完善,支持 Chrome(及 Chromium 内核浏览器,例如 Microsoft Edge、Opera、国产浏览器等)和 Firefox
    • 由于之前出现过 Chrome 商店的插件被卖掉之后加入恶意功能的先例,建议 Chrome 用户直接下载特定版本的插件并解压使用,以避免自动更新
  • Android 移动端:KeePass2Android (KP2A)
    • 开源,功能完善
    • 针对不需要软件内置同步功能的用户,有单独的 Offline 版本(不申请网络权限)
  • 同步:方案较多,比较常用的是网盘同步法
    • 借助 Dropbox、Google Drive、Microsoft OneDrive、百度网盘等客户端,将数据库文件夹映射到本地目录,然后使用桌面客户端直接将数据库保存同步,无需插件。
    • Android 客户端本身也支持部分网盘直接同步,上传、下载数据库文件。

由 LastPass 迁移到 KeePass 也很简单。以 Chrome 浏览器插件为例,插件登陆后,在菜单中 “Advanced > Export > LastPass CSV File” 可以导出 CSV 格式的密码文件。然后在 KeePass 客户端中新建一个 Database,然后在 “File > Import…” 下拉找到 LastPass CSV 即可导入之前的网站名称、账号、密码、URL 信息。