西数各系列硬盘使用 SMR 和 PMR 的型号列表

前一段时间,西数红盘系列使用 SMR(叠瓦磁记录)技术的新闻闹得很大,因为红盘 Red 系列是主打 NAS 存储,虽然不是“高端”(是与蓝盘、绿盘一样的低转速),但也比蓝盘价格贵出几成,因此被锤的很惨。

图片来源:Synology

在此之前西数是不公布每款产品的内部技术的。今天看了下西数官网,已经标出了每个系列中具体哪些型号使用 SMR、哪些使用 PMR(垂直磁记录;西数称作 CMR,常规磁记录)技术。之前传出的消息是 2~6 TB 的红盘使用 SMR,但我看了下只是一部分型号,而我在某东买的红盘在 CMR 的型号中。所以先别急着退货,可以对照下表检查一下自己的硬盘型号是否在中招的列表中,然后再做决定。

下面的型号信息综合自多个来源,仅供参考,请以官网信息为准。

SMR 技术的型号如下:

  • WD Red™ 3.5” (3.5 英寸红盘): WD20EFAX (2TB), WD30EFAX (3TB), WD40EFAX (4TB), WD60EFAX (6TB)
  • WD Blue™ 3.5” (3.5 英寸蓝盘): WD20EZAZ (2TB), WD60EZAZ (6TB)
  • WD Blue™ 2.5” (2.5 英寸蓝盘): WD10SPZX (1TB), WD20SPZX (2TB)
  • WD Black™ 2.5” (2.5 英寸黑盘): WD10SPSX (1TB)

PMR (CMR) 技术的型号如下:

  • WD Red™ (2.5 英寸 / 3.5 英寸红盘): WD10JFCX (1TB), WD10EFRX (1TB), WD20EFRX (2TB), WD30EFRX (3TB), WD40EFRX (4TB), WD60EFRX (6TB), WD80EFAX (8TB), WD100EFAX (10TB), WD101EFAX (10TB), WD120EFAX (12TB), WD140EFAX (14TB)
  • WD Red Pro (3.5 英寸红盘 Pro): WD2002FFSX (2TB), WD4002FFWX (4TB), WD4003FFBX (4TB), WD6002FFWX (6TB), WD6003FFBX (6TB), WD8003FFBX (8TB), WD102KFBX (10TB), WD121KFBX (12TB), WD141KFGX (14TB)
  • WD Black™ 3.5” (3.5 英寸黑盘): WD5003AZEX (500GB), WD1003FZEX (1TB), WD2003FZEX (2TB), WD4005FZBX (4TB), WD6003FZBX (6TB)
  • WD Black™ 2.5” (2.5 英寸黑盘): WD2500LPLX (250GB), WD3200LPLX (320GB), WD5000LPLX (500GB)
  • WD Blue™ 3.5” (3.5 英寸蓝盘): WD5000AZLX (500GB), WD5000AZRZ (500GB), WD10EZRZ (1TB), WD10EZEX (1TB), WD20EZRZ (2TB), WD30EZRZ (3TB), WD40EZRZ (4TB), WD60EZRZ (6TB)
  • WD Blue™ 2.5” (2.5 英寸蓝盘): WD3200LPCX (320GB), WD5000LPVX (500GB), WD5000LPCX (500GB), WD5000LQVX (EA) (500GB)
  • WD Purple (3.5 英寸紫盘): WD10EJRX (1TB), WD20EJRX (2TB), WD30EJRX (3TB), WD40EJRX (4TB), WD60EJRX (6TB), WD80EJRX (8TB), WD81PURZ (8TB), WD82PURZ (8TB), WD100EJRX (10TB), WD101EJRX (10TB), WD121EJRX (12TB), WD140EJRX (14TB)
  • WD Gold (3.5 英寸金盘): WD1005VBYZ (1TB), WD2005VBYZ (2TB), WD4003VRYZ (4TB), WD6003VRYZ (6TB), WD8004VRYZ (8TB), WD102VRYZ (10TB), WD121VRYZ (12TB), WD141VRYZ (14TB)

信息来源:

  1. On WD Red NAS Drives – Western Digital Blog
  2. Western Digital Online

附上西数官方博客的原文(机器翻译):

2020 年 4 月 22 日

过去的一周,可以说是多事之秋。作为一个团队,我们认真倾听并了解您对我们的WD Red NAS硬盘的反馈,特别是我们如何沟通使用哪些记录技术,这一点非常重要。我们清楚地听到了您的担忧。以下是我们通过渠道提供的客户内部硬盘的那个列表。

点击这里查看使用SMR技术的客户内部硬盘的SKU。

我们致力于提供能帮助您做出明智的购买决定的信息,并尽可能多的使用。感謝您讓我們知道如何做得更好。我们将更新我们的营销资料,并提供更多关于SMR技术的信息,包括基准和理想的用例。

再次,我们知道您将您的数据委托给我们的产品,我们不会掉以轻心。如果您已经购买了硬盘,如果您遇到性能或其他技术问题,请致电我们的客户关怀。我们将为您提供选择。我们将为您提供帮助。

更多精彩内容,敬请期待。


2020 年 4 月 20 日

最近,大家都在讨论我们的一些WD Red硬盘(HDD)中使用的记录技术。我们对任何误解表示遗憾,希望花几分钟时间讨论一下硬盘,并提供一些补充信息。

WD Red 硬盘是使用 NAS 系统的家庭和小型企业的理想选择。它们非常适合使用1到8个硬盘托架进行文件的共享和备份,一年的工作负载率为180TB,是很好的选择。我们已经对这种类型的使用进行了严格的测试,并得到了主要NAS供应商的验证。

我们通常会指定设计好的用例和性能参数,不一定要讲到引擎盖下的东西。其中一项创新技术是Shingled Magnetic Recording(SMR)技术。

SMR是经过测试和验证的技术,它使我们能够跟上个人和企业使用的数据量不断增加的趋势。我们正在不断创新,以推动它的发展。SMR 技术有不同的实现方式–硬盘管理的 SMR (DMSMR)、设备本身的 SMR (如我们的低容量 (2TB – 6TB) WD Red HDD,以及主机管理的 SMR (用于高容量数据中心应用)。每种实现都为不同的用例服务,从个人计算到世界上一些最大的数据中心都有不同的用例。

DMSMR的设计是为了管理驱动器内的智能数据放置,而不是依赖主机,从而为终端用户实现无缝集成。典型的小型企业/家庭NAS工作负载的数据强度是断断续续的,这就为DMSMR硬盘留出了足够的空闲时间,可以根据需要执行后台数据管理任务,为用户持续提供最佳的性能体验。

多年来,WD Red 硬盘已为全球各地的家庭和小型企业NAS系统提供了可靠的动力,并得到了主要NAS制造商的一致认可。在建立了这样的声誉之后,我们明白,有时我们的硬盘可能会被用于系统工作负载远远超过其预期用途。此外,有些人最近分享说,在某些数据密集型的连续读/写使用案例中,WD Red HDD 驱动的 NAS 系统的性能并不符合您的预期。

如果您遇到的性能与您的预期不符,请考虑我们为密集型工作负载设计的产品。这些产品可能包括 WD Red Pro 或 WD Gold 硬盘,或者 Ultrastar 硬盘。我们的客户服务团队随时准备提供帮助,也可以确定哪种产品可能最适合您。

我们知道您将您的数据委托给我们的产品,我们不会掉以轻心。如果您已经购买了 WD Red 硬盘,如果您遇到性能或其他技术问题,请致电我们的客户服务中心。我们将为您提供各种选择。我们将为您提供帮助。

Kubernetes + Flannel: UDP packets dropped for wrong checksum – Workaround

Recently I noticed some DNS queries in my Kubernetes cluster time out, causing apps to crash. I looked into the issue, reported to the kernel network team and applied the workaround.

Symptom

My Kubernetes cluster is built with Flannel overlay network with vxlan backend. The idea is that each node (machine) gets a private IP subnet to further allocate to pods. When a cross-node packet is to be sent, it was sent to the vxlan virtual interface, encapsulated in a UDP (regardless of the original protocol) packet and routed to the other node, where it is received by another vxlan interface and extracted.*

Kubernetes clusters provide Service resource. One of the many types of Service allows you to use a single virtual IP to represent multiple pods, some times across nodes. This is implemented with kube-proxy component, which utilizes the IPVS feature in Linux Kernel.

Now, when I make a DNS query on the host (it should be the same from inside containers, but with more hops) using dig against CoreDNS’ service IP, it always times out. It works fine if I query one of the backend pod’s IP instead.

Diagnosis

I used tcpdump to capture the packet, and noticed that the encapsulated UDP packet had a bad UDP checksum.

06:22:23.699846 IP (tos 0x0, ttl 64, id 7598, offset 0, flags [none], proto UDP (17), length 133)
    192.3.59.220.25362 > 147.135.114.20.8472: [bad udp cksum 0xd2ae -> 0x245b!] OTV, flags [I] (0x08), overlay 0, instance 1
IP (tos 0x0, ttl 63, id 33703, offset 0, flags [none], proto UDP (17), length 83)
    172.19.192.0.13169 > 172.19.195.166.53: [udp sum ok] 41922+ [1au] A? www.google.com. ar: . OPT UDPsize=4096 (55)

Further test on the receiving end shows that the packet is transferred, but dropped on the target node. That makes it certain that the checksum is what caused the DNS query to time out with no response.

A little more Googling shows that this could be caused by “Checksum offloading“. That means if the kernel wants to send a packet out on a physical ethernet card, it can leave the checksum calculation to the card hardware. In this case, if you capture the packet from kernel, it will show a wrong checksum, since it has yet to be calculated; but, the same packet captured on the receiving end will have a different and correct checksum, calculated by the sender’s network card hardware.

Workaround

I tried to use ethtool to disable TX (outgoing) checksum offloading on flannel.1 (vxlan virtual interface), and the query works again. So my guess is the kernel driver miscalculated / forgot to calculate the checksum with offloading turned on; when it’s off, it used kernel code to calculate the correct checksum before sending it to the actual outgoing network card.

To temporarily turn off checksum offloading:

sudo ethtool -K flannel.1 tx-checksum-ip-generic off

I used a systemd service to automatically do this after the interface appears. Note that the interface was created by flannel after kubelet is run, so you can’t simply execute it at boot time, e.g. in /etc/rc.local.

You can use the following code to create the service /etc/systemd/system/xiaodu-flannel-tx-off.service, then enable and start it. (The service file can be downloaded using this link.)

sudo tee /etc/systemd/system/xiaodu-flannel-tx-off.service > /dev/null << EOF
[Unit]
Description=Turn off checksum offload on flannel.1
After=sys-devices-virtual-net-flannel.1.device

[Install]
WantedBy=sys-devices-virtual-net-flannel.1.device

[Service]
Type=oneshot
ExecStart=/sbin/ethtool -K flannel.1 tx-checksum-ip-generic off
EOF
sudo systemctl enable xiaodu-flannel-tx-off
sudo systemctl start xiaodu-flannel-tx-off

For systemd >= 245, they added TransmitChecksumOffload parameter to *.link unit. You can read the docs and try it out yourself, or just use the service above.

* If you want to learn more about how Flannel and Kubernetes networking works under the hood, I strongly suggest that you read this blog post, which gives an step-by-step demonstration of how a packet is sent from one pod to another.

Solution: After switching to SyntaxHighlighter Evolved, all the codes are scrambled

tl;dr

If you switched from another syntax highlighting plugin to SyntaxHighlighter Evolved, and all your codes are scrambled, try running the following code as a single-file plugin.

function xiaodu_syntaxhighlighter_fix() {
	return 2;
}
add_filter('syntaxhighlighter_pre_getcodeformat', 'xiaodu_syntaxhighlighter_fix');

The easiest way is to go to Plugins – Plugin Editor, and paste the code at the bottom of any enabled plugin, maybe Hello Dolly.

Long version

After almost three years I finally started working on this blog again.

One of the first things I noticed is that the syntax highlighting plugin I used, Crayon Syntax Highlighter, is dead. Well, to its credit, the server-rendered markups it generated were… fine when I started this blog, but nowadays they just look sickening to me, especially when compared to the neat client rendering solutions.

So I went to the store and downloaded the most popular choice, SyntaxHighlighter Evolved, which uses the JavaScript library SyntaxHighlighter to perform client-side highlighting. After installing it and converting all my old code tags to their markup format, I found the highlighting to be working, but all the C++ and HTML looked screwed up.

Scrambled code
Scrambled code

As you can see, all the “<“, “>” and “&” in the code are now showing up as their HTML entities – “&lt;”, “&rt;” and “&amp;”. That is not cool, so I looked into the problem.

Looking under the hood

First thing we need to know is how the code is stored in the post. When I click on the “Text” tab in the post editor (yep, the old one… I haven’t adapted to the new blocks yet,) I found that the characters are displaying correctly.

Code in post editor
Code in post editor

Then I looked further into MySQL, and the code is stored encoded, which is fine – it can be stored in the database however it fits, as long as the final output is correct… which it isn’t.

Code in MySQL
Code in MySQL

Pinpointing the plugin code

Now that I know that the plugin did an extra encoding, I looked for “htmlspecialchars” in the plugin’s GitHub repository, and found this piece of code:

	// This function determines what version of SyntaxHighlighter was used when the post was written
	// This is because the code was stored differently for different versions of SyntaxHighlighter
	function get_code_format( $post ) {
		if ( false !== $this->codeformat )
			return $this->codeformat;
		if ( empty($post) )
			$post = new stdClass();
		if ( null !== $version = apply_filters( 'syntaxhighlighter_pre_getcodeformat', null, $post ) )
			return $version;
		$version = ( empty($post->ID) || get_post_meta( $post->ID, '_syntaxhighlighter_encoded', true ) || get_post_meta( $post->ID, 'syntaxhighlighter_encoded', true ) ) ? 2 : 1;
		return apply_filters( 'syntaxhighlighter_getcodeformat', $version, $post );
	}
	// Adds a post meta saying that HTML entities are encoded (for backwards compatibility)
	function mark_as_encoded( $post_ID, $post ) {
		if ( false == $this->encoded || 'revision' == $post->post_type )
			return;
		delete_post_meta( $post_ID, 'syntaxhighlighter_encoded' ); // Previously used
		add_post_meta( $post_ID, '_syntaxhighlighter_encoded', true, true );
	}

Apparently years ago they changed how codes are stored in the post. Now if you write and save a new post with their plugin installed, they will save the code already encoded, and insert a post meta “_syntaxhighlighter_encoded = True” to mark the post as the “new (encoded) format”.

But if you are like me who used other plugins when initially posting the code and later switched to Evolved, you are in bad luck, as they consider your post by default the “old format,” and will encode the code again in the final output.

Solution

The apparent solutions it to make the plugin think that all my posts are in the new format. I could add the same metadata to each of the posts, but luckily there is a easier way: Use the filter “syntaxhighlighter_pre_getcodeformat” (line 8 in the code above) they provided to override the result.

So I used the plugin code at the beginning to hook it. The hook function simply returns 2, which means all my posts, with or without the metadata, will be considered the already-encoded new format, so they will not be doubly encoded.

It’s been years, is that all you have to say?

OK, fair enough. So this blog may look the same, but the tech underneath it is constantly changing.

For example, all my appliances are now hosted on my own bare-metal (as opposed to cloud-vendored like GKE) globally-distributed Kubernetes cluster. Also, I have been hiding behind CloudFlare for years to avoid the haters, but now they are mostly gone (or grown up 🙄,) so I have been thinking of new ways to distribute my content.

All of these new stuff are exciting and worth sharing, and I will write about them soon™.

记一次杯具与恢复的全过程(安卓修改屏幕分辨率)

我手里有一个三星的安卓平板(SM-P601),分辨率是 2560×1600,像素密度 320,经常被某些傻X APP 认为不是移动设备。今天刚发现可以通过一个 wm 命令来临时修改分辨率,于是手贱改了个 640×360,然后悲剧发生了:因为我是在 Terminal Emulator 里 Root 修改的,没有开 ADB,然后屏幕小到看不全,也没法打 wm reset,重启也不管用。借助万能的谷歌,终于坎坷的恢复回去了。下面记录一下各种坑……

第一关:怎么启用 ADB?

在网上搜了一下,果然有蛋疼的老外跟我一样蠢,在没开 ADB 的情况下修改了分辨率,还改不回去了,例如 XDA 的这位仁兄,然后他找出了如何通过 Recovery 强制启用 ADB 的方法:

我用的是 TWRP,首先长按电源键重启,并按住 Home + 音量加进入 Recovery(不同设备组合键也不一样)。然后在 Mount 中挂载 System,再进入 Advanced – Terminal,并使用 vi /system/build.prop 加入以下行:

persist.service.adb.enable=1                                                    
persist.service.debuggable=1
persist.sys.usb.config=mtp,adb

如果不会使用 vi 或者觉得麻烦,可以用 File Manager 把 /system/build.prop 拷到 /sdcard,然后利用 TWRP 的 MTP 把它复制到电脑上修改,然后再拷贝回去。

重启系统后,电脑上使用 adb devices 可以看到设备了,然而后面显示 unauthorized,无法进入 adb shell……于是进入第二关。(如果你的设备版本较低无需 ADB 授权,或以前授权过这台电脑,可以直接跳到最后。)

第二关:怎么授权 ADB?

还是没办法进行屏幕操作。一般连上 ADB 之后都会有个“是否允许 USB 调试”的提示,点允许就授权好了。于是又放狗一搜,果然也有 StackOverflow 的老外解决了这个问题。

首先找到你电脑上的 adbkey.pub 公钥,位置可能在 $ANDROID_SDK_HOME 环境变量对应的位置(如果安装过 SDK)、C:\Users\用户名\.android(Windows 默认)或 ~/.android(其它系统默认)。找到之后,改名为 adb_keys,并利用 TWRP 的 MTP 传输到手机的 /sdcard(也就是内部存储)根目录。

然后回到 TWRP 的 Advanced – File Manager,找到 /sdcard/adb_keys 并复制到 /data/misc/adb 目录下。重启手机即可连接 ADB。

如何正确地修改分辨率?

在系统中进入 adb shell,在修改分辨率时要保证屏幕是关闭(锁屏)状态,所以最好不要在 Terminal Emulator 命令行中修改。

要恢复默认的分辨率,根据系统版本不同对应的命令是:

  • am display-size reset(< Android 4.3)
  • wm size reset(≥ Android 4.3)

然后如果要修改分辨率或密度(density),使用的命令是:

  • am display-size 1080x720
  • am display-density 240
  • wm size 1080x720
  • wm density 240

ngx_pagespeed module for Ubuntu’s stock nginx

I recently decided to build ngx_pagespeed for Ubuntu’s stock nginx, since nginx supports dynamic modules (as Apache did long ago) since 1.9.11. Being “dynamic” means that third-party modules don’t have to be built into the main nginx binary, instead can be separately built and dynamically load at runtime, as long as the module and main nginx are compatible, meaning their configuration parameters are similar enough.

(中文版请参见另一篇文章:为 Ubuntu 官方源的 nginx 单独编译 ngx_pagespeed 模块

I have not seen anyone else build any third-party modules separately. nginx.org official repository provides several modules as packages, but I believe they are built along with the binary they shipped. A probable reason is that even though modules built separately can be loaded, they are not guaranteed to work smoothly. I saw that nginx 1.11.5 is trying to improve compatibility of dynamic modules, but whether that works remains to be tested.

I used the latest Debian (Ubuntu) packaging tools and formats to build the package, and Launchpad PPA to host the apt repository.

You can use the apt commands shown on the page to enable the PPA and install the packages. The following is an example:

sudo add-apt-repository ppa:du9l/ngx-pagespeed
sudo apt-get update

To use this module, first install the right flavored package, then add the line:
load_module modules/ngx_pagespeed.so;
… to your /etc/nginx/nginx.conf file before reloading nginx.

Please remember that this is an experimental feature, and you should NOT use it in production. More information can be found in the PPA description below.

PPA Description

This repository contains “ngx_pagespeed” dynamic module for Ubuntu’s stock nginx packages, including all flavors available in the official repository.

* WARNING: Building dynamic modules alone is EXPERIMENTAL. It is NOT guaranteed to work by the nginx authors. Even though the module can be loaded and has been tested on my own server, I still don’t recommend using it in PRODUCTION environments. USE AT YOUR OWN RISK. *

The package names are “ngx-pagespeed-nginx-FLAVOR” where FLAVOR is core / light / full / extras, which should match your nginx flavor. Check with:
$ dpkg -l | grep nginx

The versions follow ngx_pagespeed’s latest stable versions and Ubuntu’s REL-updates (e.g. xenial-updates) nginx versions. For example, the first version available in this PPA is built with ngx_pagespeed 1.11.33.4 and xenial-updates’ nginx 1.10.0-0ubuntu0.16.04.4 versions.

To use this module, first install the right flavored package, and then add the line:
load_module modules/ngx_pagespeed.so;
… to your /etc/nginx/nginx.conf file before reloading nginx.

* NOTE: This package is linked against the pre-built “psol” binaries provided by Google, so only i386 and amd64 systems are supported for now. In the future I will update the package to build psol itself. *