flickr对javascript干的好事
发表于2006-11-14 17:23:48

在一个讨论web技术的网站vitamin上发现这篇《Serving JavaScript Fast》,读过之后大有收获,茅塞顿开。

先在网上查了查,已经有blog谈到这篇文章(我算是后知后觉了),

有总结要点的《Flickr 的开发者的 Web 应用优化技巧》,也有延伸开来的《接着讲Flickr的八卦》

但似乎没有全文翻译的(这下就好,不会忙了半天发现是无用功)。

之后,就写信问作者可不可以,作者一口答应:“sure - i’d love you to translate it”

,只是要求我翻好之后给他一个链接地址。得到准许,心里就有底了。

先介绍一下作者。Cal Henderson,伦敦人,现居加利福尼亚的旧金山。

PHP,MySQL和Perl专家,现任flickr架构师

(flickr被收购后就在yahoo了),同时也是vitamin的特聘顾问(写些技术性文章)。

既然他是架构师,flickr用的应该就是文中谈到的这些技术,于是参照文章,再对比网站,

种种迹象表明确实如此。

虽然在中国访问flickr速度不敢恭维,加速效果不得而知,

但其用了n多css和javascript资源却似乎从没出过什么问题,

也从侧面印证了这些技术的有效性。

先在网上查了查,已经有blog谈到这篇文章(我算是后知后觉了),

有总结要点的《Flickr 的开发者的 Web 应用优化技巧》

也有延伸开来的《接着讲Flickr的八卦》,但似乎没有全文翻译的(这下就好,不会忙了半天发现是无用功)。

之后,就写信问作者可不可以,作者一口答应:“sure - i’d love you to translate it”,

只是要求我翻好之后给他一个链接地址。得到准许,心里就有底了。

先介绍一下作者。Cal Henderson,伦敦人,现居加利福尼亚的旧金山。

 

PHP,MySQL和Perl专家,现任flickr架构师(flickr被收购后就在yahoo了),

 

同时也是vitamin的特聘顾问(写些技术性文章)。

既然他是架构师,flickr用的应该就是文中谈到的这些技术,于是参照文章,再对比网站,

 

种种迹象表明确实如此。虽然在中国访问flickr速度不敢恭维,加速效果不得而知,

 

但其用了n多css和javascript资源却似乎从没出过什么问题,也从侧面印证了这些技术的有效性。

仔细的看完文章,还有个强烈的感觉:这老兄也太能卖关子了,一句话非分成三句说,

 

摆事实讲道理是够透彻,就是有点太@#$%了…… 算了,

 

他怎么说我怎么翻吧,忠实于原著嘛,要不就成篡改了。

 

经过几天努力,加上同事thincat兄倾力援手(小弟不胜感激啊),

 

终于完工(@_@ 真是苦力活啊,我再也不想干了~)。

全文翻译如下:

让javascript跑得更快

作者:Cal Henderson

下一代web应用让javascript和css得堪大用。我们会告诉你怎样使这些应用又快又灵。

建立了号称“Web 2.0”的应用,也实现了富内容(rich content)和交互,

 

我们期待着css和javascript扮演更加重要的角色。为使应用干净利落,我们需要完善那些渲染页面的文件,

 

优化其大小和形态,以确保提供最好的用户体验——在实践中,

 

这就意味着一种结合:使内容尽可能小、下载尽可能快,同时避免对未改动资源不必要的重新获取。

由于css和js文件的形态,情况有点复杂。跟图片相比,其源代码很有可能频繁改动。

 

而一旦改动,就需要客户端重新下载,使本地缓存无效(保存在其他缓存里的版本也是如此)。

 

在这篇文章里,我们将着重探讨怎样使用户体验最快:包括初始页面的下载,随后页面的下载,

 

以及随着应用渐进、内容变化而进行的资源下载。

我始终坚信这一点:对开发者来说,应该尽可能让事情变得简单。

 

所以我们青睐于那些能让系统自动处理优化难题的方法。

 

只需少许工作量,我们就能建立一举多得的环境:它使开发变得简单,有极佳的终端性能,

 

也不会改变现有的工作方式。

好大一沱

老的思路是,为优化性能,可以把多个css和js文件合并成极少数大文件。

 

跟十个5k的js文件相比,合并成一个50k的文件更好。虽然代码总字节数没变,

 

却避免了多个HTTP请求造成的开销。每个请求都会在客户端和服务器两边有个建立和消除的过程,

 

导致请求和响应header带来开销,还有服务器端更多的进程和线程资源消耗

 

(可能还有为压缩内容耗费的cpu时间)。

(除了HTTP请求,)并发问题也很重要。默认情况下,在使用持久连接(persistent connections)时,

 

ie和firefox在同一域名内只会同时下载两个资源(在HTTP 1.1规格书中第8.1.4节的建议)

 

(htmlor注:可以通过修改注册表等方法改变这一默认配置)。

 

这就意味着,在我们等待下载2个js文件的同时,

 

将无法下载图片资源。也就是说,这段时间内用户在页面上看不到图片。

(虽然合并文件能解决以上两个问题,)可是,这个方法有两个缺点。

 

第一,把所有资源一起打包,将强制用户一次下载完所有资源。

 

如果(不这么做,而是)把大块内容变成多个文件,下载开销就分散到了多个页面,

 

同时缓解了会话中的速度压力(或完全避免了某些开销,这取决于用户选择的路径)。

 

如果为了随后页面下载得更快而让初始页面下载得很慢,

我们将发现更多用户根本不会傻等着再去打开下一个页面。

第二(这个影响更大,一直以来却没怎么被考虑过),在一个文件改动很频繁的环境里,

 

如果采用单文件系统,那么每次改动文件都需要客户端把所有css和js重新下载一遍。

 

假如我们的应用有个100k的合成的js大文件,任何微小的改动都将强制客户端把这100k再消化一遍。

分解之道

(看来合并成大文件不太合适。)替代方案是个折中的办法:把css和js资源分散成多个子文件,

 

按功能划分、保持文件个数尽可能少。这个方案也是有代价的,

 

虽说开发时代码分散成逻辑块(logical chunks)能提高效率,可在下载时为提高性能还得合并文件。

 

不过,只要给build系统(把开发代码变成产品代码的工具集,是为部署准备的)加点东西,就没什么问题了。

对于有着不同开发和产品环境的应用来说,用些简单的技术可以让代码更好管理。

 

在开发环境下,为使条理清晰,代码可以分散为多个逻辑部分(logical components)。

 

可以在Smarty(一种php模板语言)里建立一个简单的函数来管理javascript的下载:

SMARTY:
{insert_js files="foo.js,bar.js,baz.js"}

PHP:
function smarty_insert_js($args){
  foreach (explode(',', $args['files']) as $file){
    echo "<script type=\"text/javascript\" SOURCE=\"/javascript/$file\"></script>\n";
  }
}

OUTPUT:
<script type="text/javascript" SOURCE="/javascript/foo.js"></script>
<script type="text/javascript" SOURCE="/javascript/bar.js"></script>
<script type="text/javascript" SOURCE="/javascript/baz.js"></script>

(htmlor注:wordpress中会把“src”替换成不知所谓的字符,因此这里只有写成“SOURCE”,

 

使用代码时请注意替换,下同)

就这么简单。然后我们就命令build过程(build process)去把确定的文件合并起来。

 

这个例子里,合并的是foo.js和bar.js,因为它们几乎总是一起下载。

 

我们能让应用配置记住这一点,并修改模板函数去使用它。(代码如下:)

SMARTY:
{insert_js files="foo.js,bar.js,baz.js"}

PHP:
# 源文件映射图。在build过程合并文件之后用这个图找到js的源文件。

$GLOBALS['config']['js_source_map'] = array(
  'foo.js'	=> 'foobar.js',
  'bar.js'	=> 'foobar.js',
  'baz.js'	=> 'baz.js',
);

function smarty_insert_js($args){
  if ($GLOBALS['config']['is_dev_site']){
    $files = explode(',', $args['files']);
  }else{
    $files = array();
    foreach (explode(',', $args['files']) as $file){
      $files[$GLOBALS['config']['js_source_map'][$file]]++;
    }
    $files = array_keys($files);
  }

  foreach ($files as $file){
    echo "<script type=\"text/javascript\" SOURCE=\"/javascript/$file\"></script>\n";
  }
}

OUTPUT:
<script type="text/javascript" SOURCE="/javascript/foobar.js"></script>
<script type="text/javascript" SOURCE="/javascript/baz.js"></script>

模板里的源代码没必要为了分别适应开发和产品阶段而改动,它帮助我们在开发时保持文件分散,

 

发布成产品时把文件合并。想更进一步的话,可以把合并过程(merge process)写在php里,

 

然后使用同一个(合并文件的)配置去执行。这样就只有一个配置文件,避免了同步问题。

 

为了做的更加完美,我们还可以分析css和js文件在页面中同时出现的几率,

 

以此决定合并哪些文件最合理(几乎总是同时出现的文件是合并的首选)。

对css来说,可以先建立一个主从关系的模型,它很有用。一个主样式表控制应用的所有样式表,

 

多个子样式表控制不同的应用区域。采用这个方法,大多数页面只需下载两个css文件,

 

而其中一个(指主样式表)在页面第一次请求时就会缓存。

对没有太多css和js资源的应用来说,这个方法在第一次请求时可能比单个大文件慢,

 

但如果保持文件数量很少的话,你会发现其实它更快,因为每个页面的数据量更小。

 

让人头疼的下载花销被分散到不同的应用区域,因此并发下载数保持在一个最小值,

 

同时也使得页面的平均下载数据量很小。

压缩

谈到资源压缩,大多数人马上会想到mod_gzip(但要当心,mod_gzip实际上是个魔鬼,至少能让人做恶梦)。

 

它的原理很简单:浏览器请求资源时,会发送一个header表明自己能接受的内容编码。就像这样:

Accept-Encoding: gzip,deflate

服务器遇到这样的header请求时,就用gzip或deflate压缩内容发往客户端,然后客户端解压缩。

 

这过程减少了数据传输量,同时消耗了客户端和服务器的cpu时间。也算差强人意。

 

但是,mod_gzip的工作方式是这样的:先在磁盘上创建一个临时文件,然后发送(给客户端),

 

最后删除这个文件。在高容量的系统中,由于磁盘io问题,很快就会达到极限。

 

要避免这种情况,可以改用mod_deflate(apache 2才支持)。

 

它采用更合理的方式:在内存里做压缩。对于apache 1的用户来说,可以建立一块ram磁盘,

 

让mod_gzip在它上面写临时文件。虽然没有纯内存方式快,但也不会比往磁盘上写文件慢。

话虽如此,其实还是有办法完全避免压缩开销的,那就是预压缩相关静态资源,

 

下载时由mod_gzip提供合适的压缩版本。如果把压缩添加在build过程,它就很透明了。

 

需要压缩的文件通常很少(用不着压缩图片,因为并不能减小更多体积),

 

只有css和js文件(和其他未压缩的静态内容)。

配置选项会告诉mod_gzip去哪里找到预压缩过的文件。

mod_gzip_can_negotiate	Yes
mod_gzip_static_suffix	.gz
AddEncoding	gzip	.gz

新一点的mod_gzip版本(从1.3.26.1a开始)添加一个额外的配置选项后,

 

就能自动预压缩文件。不过在此之前,必须确认apache有正确的权限去创建和覆盖压缩文件。

mod_gzip_update_static	Yes

可惜,事情没那么简单。某些Netscape 4的版本(尤其是4.06-4.08)认为自己能够解释压缩内容

 

(它们发送一个header这么说来着),但其实它们不能正确的解压缩。

 

大多数其他版本的Netscape 4在下载压缩内容时也有各种各样的问题。

 

所以要在服务器端探测代理类型,(如果是Netscape 4,就要)让它们得到未压缩的版本。

 

这还算简单的。ie(版本4-6)有些更有意思的问题:当下载压缩的javascript时,

 

有时候ie会不正确的解压缩文件,或者解压缩到一半中断,然后把这半个文件显示在客户端。

 

如果你的应用对javascript的依赖比较大(htmlor注:比如ajax应用),那么就得避免发送压缩文件给ie。

 

在某些情况下,一些更老的5.x版本的ie倒是能正确的收到压缩的javascript,

 

可它们会忽略这个文件的etag header,不缓存它。

 

(thincat友情提示:尽管压缩存在一些浏览器不兼容的现象,

 

由于这些不能很好的支持压缩的浏览器数量现在已经非常少了,

 

我认为这种由于浏览器导致的压缩不正常的情况可以忽略不计。

 

这些过时的浏览器还能不能在现在流行的windows或unix环境下面安装都存在不小的问题)

既然gzip压缩有这么多问题,我们不妨把注意力转到另一边:不改变文件格式的压缩。

 

现在有很多这样的javascript压缩脚本可用,大多数都用一个正则表达式驱动的语句集来减小源代码的体积。

 

它们做的不外乎几件事:去掉注释,压缩空格,缩短私有变量名和去掉可省略的语法。

不幸的是,大多数脚本效果并不理想,要么压缩率相当低,

 

要么某种情形下会把代码搞得一团糟(或者两者兼而有之)。

 

由于对解析树的理解不完整,压缩器很难区分一句注释和一句看似注释的引用字符串。

 

因为闭合结构的混合使用,要用正则表达式发现哪些变量是私有的并不容易,

 

因此一些缩短变量名的技术会打乱某些闭合代码。

还好有个压缩器能避免这些问题:dojo压缩器(现成的版本在这里)。

 

它使用rhino(mozilla的javascript引擎,是用java实现的)建立一个解析树,

 

然后将其提交给文件。它能很好的减小代码体积,

 

仅用很小的成本:因为只在build时压缩一次。由于压缩是在build过程中实现的,

 

所以一清二楚。(既然压缩没有问题了,)我们可以在源代码里随心所欲的添加空格和注释,

 

而不必担心影响到产品代码。

与javascript相比,css文件的压缩相对简单一些。由于css语法里不会有太多引用字符串

 

(通常是url路径跟字体名),我们可以用正则表达式大刀阔斧的干掉空格

 

(htmlor注:这句翻的最爽,哈哈)。如果确实有引用字符串的话,我们总可以把一串空格合成一个

 

(因为不需要在url路径和字体名里查找多个空格和tab)。这样的话,一个简单的perl脚本就够了:

___FCKpd___5

然后,就可以把单个的css文件传给脚本去压缩了。命令如下:

perl compress.pl site.source.css > site.compress.css

做完这些简单的纯文本优化工作后,我们就能减少数据传输量多达50%了(这个量取决于你的代码格式,可能更多)。

 

这带来了更快的用户体验。不过我们真正想做的是,尽可能避免用户请求的发生——除非确实有必要。

 

这下HTTP缓存知识派上用场了。

 

缓存真是个好东西

看来我们正从错误的方向入手解决问题。(现在的问题是,)这些可能的缓存策略导致了一件事情反复发生,

 

那就是:客户端向服务器查询本地缓存是否最新。假如服务器在改动文件的时候通知客户端,

 

客户端不就知道它的缓存是最新的了(直到接到下一次通知)?可惜天公不做美——(事实)

 

是客户端向服务器发出请求。

其实,也不尽然。在获取js或css文件之前,客户端会用<script>或<link>标记向服务器发送一个请求,

 

说明哪个页面要加载这些文件。这时候就可以用服务器的响应来通知客户端这些文件有了改动。

 

有点含糊,说得再详细点就是:如果改变css和js文件内容的同时,也改变它们的文件名,

 

就可以告诉客户端对url全都永久缓存——因为每个url都是唯一的。

假如能确定一个资源永不更改,我们就可以发出一些霸气十足的缓存header(htmlor注:这句也很有气势吧)。

 

在php里,两行就好:

<?php
header("Expires: ".gmdate("D, d M Y H:i:s", time()+315360000)." GMT");
header("Cache-Control: max-age=315360000");
?>

我们告诉浏览器这个内容在10年后(10年大概会有315,360,000秒,或多或少)过期,

 

浏览器将会保留它10年。当然,很有可能不用php输出css和js文件(因此就不能发出header),

 

这种情况将在稍后说明。

人力有时而穷

当文件内容更改时,手动去改文件名是很危险的。假如你改了文件名,模板却没有指向它?

 

假如你改了一些模板另一些却没改?假如你改了模板却没改文件名?还有最糟的,

 

假如你改动了文件却忘了改名或者忘了改变对它的引用?最好的结果,是用户看到老的而看不到新的内容。

 

最坏的结果,是找不到文件,网站没法运转了。听起来这(指改动文件内容时修改url)似乎是个馊主意。

幸运的是,计算机做这类事情——当某种变化发生,需要相当准确地完成的、

 

重复重复再重复的(htmlor注:番茄鸡蛋伺候~)、枯燥乏味的工作——总是十分在行。

这个过程(改变文件的url)没那么痛苦,因为我们根本不需要改文件名。

 

资源的url和磁盘上文件的位置也没必要保持一致。

 

使用apache的mod_rewrite模块,可以建立简单的规则,让确定的url重定向到确定的文件。

RewriteEngine on
RewriteRule ^/(.*.)v[0-9.]+.(css|js|gif|png|jpg)$	/$1$2	[L]

这条规则匹配任何带有指定扩展名同时含有“版本”信息(version nugget)的url,

 

它会把这些url重定向到一个不含版本信息的路径。如下所示:

URL			   Path
/images/foo.v2.gif	-> /images/foo.gif
/css/main.v1.27.css	-> /css/main.css
/javascript/md5.v6.js	-> /javascript/md5.js

使用这条规则,就可以做到不改变文件路径而更改url(因为版本号变了)。

 

由于url变了,浏览器就认为它是另一个资源(会重新下载)。想更进一步的话,

 

可以把我们之前说的脚本编组函数结合起来,根据需要生成一个带有版本号的<script>标记列表。

说到这里,你可能会问我,为什么不在url结尾加一个查询字符串(query string)呢

 

(如/css/main.css?v=4)?根据HTTP缓存规格书所说,用户代理对含有查询字符串的url永不缓存。

 

虽然ie跟firefox忽略了这点,opera和safari却没有——为了确保所有浏览器都缓存你的资源,

 

还是不要在url里用查询字符串的好。

现在不移动文件就能更改url了,如果能让url自动更新就更好了。

 

在小型的产品环境下(如果有大型的产品环境,就是开发环境了),

 

使用模板功能可以很轻易的实现这点。这里用的是smarty,用其他模板引擎也行。

SMARTY:
<link xhref="{version xsrc='/css/group.css'}" rel="stylesheet" type="text/css" />

PHP:
function smarty_version($args){
  $stat = stat($GLOBALS['config']['site_root'].$args['src']);
  $version = $stat['mtime'];

  echo preg_replace('!.([a-z]+?)$!', ".v$version.$1", $args['src']);
}

OUTPUT:
<link xhref="/css/group.v1234567890.css" mce_href="/css/group.v1234567890.css" 
rel="stylesheet" type="text/css" />

对每个链接到的资源文件,我们得到它在磁盘上的路径,检查它的mtime(文件最后修改的日期和时间),

然后把这个时间当作版本号插入到url中。对于低流量的站点(它们的stat操作开销不大)或者开发环境来说,

 

这个方案不错,但对于高容量的环境就不适用了——因为每次stat操作都要磁盘读取(导致服务器负载升高)。

解决方案相当简单。在大型系统中每个资源都已经有了一个版本号,就是版本控制的修订号

 

(你们应该使用了版本控制,对吧?)。当我们建立站点准备部署的时候,可以轻易的查到每个文件的修订号,

 

写在一个静态配置文件里。

<?php
$GLOBALS['config']['resource_versions'] = array(
  '/images/foo.gif'    => '2.1',
  '/css/main.css'      => '1.27',
  '/javascript/md5.js' => '6.1.4',
);
?>

当我们发布产品时,可以修改模板函数来使用版本号。

<?php
function smarty_version($args){
  if ($GLOBALS['config']['is_dev_site']){
    $stat = stat($GLOBALS['config']['site_root'].$args['src']);
    $version = $stat['mtime'];
  }else{
    $version = $GLOBALS['config']['resource_versions'][$args['src']];
  }

  echo preg_replace('!.([a-z]+?)$!', ".v$version.$1", $args['src']);
}
?>

就这样,不需要改文件名,也不需要记住改了哪些文件——当文件有新版本发布时它的url就会

 

自动更新——有意思吧?我们就快搞定了。

只欠东风

之前谈到为静态文件发送超长周期(very-long-period)的缓存header时曾说过,

 

如果不用php输出,就不能轻易的发送缓存header。很显然,

 

有两个办法可以解决:用php输出,或者让apache来做。

php出马,手到擒来。我们要做的仅仅是改变rewrite规则,把静态文件指向php脚本,

 

用php在输出文件内容之前发送header。

SMARTY:
{insert_js files="foo.js,bar.js,baz.js"}

PHP:
function smarty_insert_js($args){
  foreach (explode(',', $args['files']) as $file){

   echo "<script type=\"text/javascript\" src=\"/javascript/$file\"></script>\n";
  }
}

OUTPUT:
<script type="text/javascript" src="/javascript/foo.js"></script>
<script type="text/javascript" src="/javascript/bar.js"></script>
<script type="text/javascript" src="/javascript/baz.js"></script>

这个方案有效,但并不出色。(因为)跟apache相比,php需要更多内存和执行时间。

 

另外,我们还得小心防止可能由path参数传递伪造值引起的exploits。

 

为避免这些问题,应该用apache直接发送header。

 

rewrite规则语句允许当规则匹配时设置环境变量(environment variable),

 

当给定的环境变量设置后,Header命令就可以添加header。

 

 

结合以下两条语句,我们就把rewrite规则和header设置绑定在了一起:

SMARTY:
{insert_js files="foo.js,bar.js,baz.js"}

PHP:
# map of where we can find .js source files after the build process
# has merged as necessary

$GLOBALS['config']['js_source_map'] = array(
  'foo.js'	=> 'foobar.js',
  'bar.js'	=> 'foobar.js',
  'baz.js'	=> 'baz.js',
);

function smarty_insert_js($args){

  if ($GLOBALS['config']['is_dev_site']){

    $files = explode(',', $args['files']);
  }else{

    $files = array();

    foreach (explode(',', $args['files']) as $file){

      $files[$GLOBALS['config']['js_source_map'][$file]]++;
    }

    $files = array_keys($files);
  }

  foreach ($files as $file){

   echo "<script type=\"text/javascript\" src=\"/javascript/$file\"></script>\n";
  }
}

OUTPUT:
<script type="text/javascript" src="/javascript/foobar.js"></script>
<script type="text/javascript" src="/javascript/baz.js"></script>

考虑到apache的执行顺序,应该把rewrite规则加在主配置文件(httpd.conf)

 

而不是目录配置文件(.htaccess)中。否则在环境变量设置之前,header行会先执行(就那没意义了)。

 

至于header行,则可以放在两文件任何一个当中,没什么区别。

眼观六路

(htmlor注:多谢tchaikov告知“skinning rabbits”的含义,但我不想翻的太正式,

 

眼下的这个应该不算太离谱吧。)

通过结合使用以上技术,我们可以建立一个灵活的开发环境和一个快速又高性能的产品环境。

 

当然,这离终极目标“速度”还有一段距离。有许多更深层的技术(比如分离伺服静态内容,

 

用多域名提升并发量等)值得我们关注,包括与我们谈到的方法(建立apache过滤器,修改资源url,

 

加上版本信息)殊途同归的其他路子。你可以留下评论,告诉我们那些你正在使用的卓有成效的技术和方法。

(完)

The next generation of web apps make heavy use of JavaScript and CSS. We’ll show you how to make those apps responsive and quick.

With our so-called "Web 2.0" applications and their rich content and interaction, we expect our applications to increasingly make use of CSS and JavaScript. To make sure these applications are nice and snappy to use, we need to optimize the size and nature of content required to render the page, making sure we’re delivering the optimum experience. In practice, this means a combination of making our content as small and fast to download as possible, while avoiding unnecessarily refetching unmodified resources.

This is complicated a little by the nature of CSS and JavaScript resources. In contrast to image assets, CSS and JavaScript source code is very likely to change many times as time goes by. When these resources change, we need our clients to download them all over again, invalidating the version in their local cache (and any versions stored in other caches along the way). In this article, we’ll look at ways we can make the whole experience as fast as possible for our users - the initial page load, subsequent page loads and ongoing resource loading as the application evolves and content changes.

I believe strongly in making things as simple as possible for developers, so we’ll also be looking at ways we can set up our systems to automatically take care of these optimization issues for us. With a little up front work, we can get the best of both worlds - an environment that makes development easy with great end-user performance - all without changing the way we work.

Monolith

The old school of thought was that we could achieve optimal performance by combining multiple CSS and JavaScript files into fewer, larger blocks. Rather than having ten 5k JavaScript files, we combine them into a single 50k file. While the total size of the code is still the same, we avoid having the overhead associated with multiple HTTP requests. Each request has a setup and teardown phase on both the client and server, incurs request and response header size overhead, and resource overhead on the server side in the form of more processes or threads (and perhaps more CPU time for on-the-fly gzipped content).

The parellization aspect is also important. By default, both Internet Explorer and Mozilla/Firefox will only download two resources from a single domain at once when using persistent connections (as suggested in the HTTP 1.1 spec, section 8.1.4). This means that while we’re waiting to download those JavaScript files, 2 at a time, we’re not loading image assets - the page our users see during the loading phase will be missing its images.

However, there are a couple of downsides to this approach. By bundling all of our resources together, we force the user to download everything up front. By chunking content into multiple files we can spread out the cost of loading across several pages, amortizing the speed hit across a session (or avoiding some of the cost completely, depending on the path the user chooses). If we make the first page slow to speed up subsequent pages, we might find that we have more users who never wait around to request a second page.

The big downside to the single file approach has not often, historically, been considered. In an environment where we will have to often change our resources, any changes to a single-file system will require the client to re-download a copy of the entire CSS or JavaScript working set. If our application has a single monolithic 100k JavaScript source file, any tiny change to our code will force all clients to suck down the 100k all over again.

A splintered approach

The alternative approach lies somewhere in the middle - we split our CSS and JavaScript resources into multiple sub-files, while at the same time keeping that number functionally low. This compromise comes at a cost - we need to be able to develop applications with our code split out into logical chunks to increase development efficiency, while delivering merged files for performance. With a few additions to our build system (the set of tools which turn your development code into production code, ready for deployment), this needn’t be a compromise we have to make.

For an application environment with distinct development and production environments, you can use a few simple techniques to keep your code manageable. In your development environment, code can be split into many logical components to make separation clear. In Smarty (A PHP templating language) we can create a simple function to manage the loading of our JavaScript:

___FCKpd___16

So far, so easy. But then we instruct our build process to merge certain files together into single resources. In our example, imagine we merged foo.js and bar.js into foobar.js, since they are nearly always loaded together. We can then record this fact in our application configuration and modify our template function to use this information.

___FCKpd___17

The source code in our templates doesn’t need to change between development and production, but allows us to keep files separated while developing and merged in production. For bonus points, we can write our merging process in PHP and use the same configuration block to perform the merge process, allowing us to keep a single configuration file and avoid having to keep anything in sync. For super-bonus points, we could analyze the occurrence of scripts and style sheets together on pages we serve, to determine which files would be best to merge (files that nearly always appear together are good candidates for merging).

For CSS, a useful model to start from is that of a master and subsection relationship. A single master style sheet controls style across your entire application, while multiple sub-sheets control various distinct feature areas. In this way, most pages will load only two sheets, one of which is cached the first time any page is requested (the master sheet).

For small CSS and JavaScript resource sets, this approach may be slower for the first request than a single large resource, but if you keep the number of components low then you’ll probably find it’s actually faster, since the data size per page is much lower. The painful loading costs are spread out around different application areas, so the number of parallel loads is kept to a minimum while also keeping the resources-per-page size low.

Compression

When talk about asset compression, most people think immediately of mod_gzip. Beware, however - mod_gzip is actually evil, or at the least, a resource hogging nightmare. The idea behind it is simple - browsers request resources and send along a header to show what kind of content encodings they accept. It looks something like this:

Accept-Encoding: gzip,deflate

When a server encounters this header, it can then gzip or deflate (compress) the content it’s sending to the client, where the client will then decompress it. This burns CPU time on both the client and server, while reducing the amount of data transferred. All well and good. The way mod_gzip works, however, is to create a temporary file on disk in which to compress the source data, serve that file out, then delete it. For high volume systems, you very quickly become bound by disk IO. We can avoid this by using mod_deflate instead (Apache 2 only), which does all the compression in memory - sensible. For Apache 1 users, you can instead create a RAM disk and have mod_gzip writes its temporary files there - not quite as fast as pure in-memory compression, but not nearly as slow as writing to disk.

Even so, we can avoid the compression overhead completely by pre-compressing the relevant static resources and using mod_gzip to serve people the compressed version where appropriate. If we add this compression into our build process, it all happens transparently to us. The number of files that need compressing is typically quite low - we don’t compress images since we don’t gain much, if any, size benefit (since they’re already compressed) so we only need to compress our JavaScript and CSS (and any other uncompressed static content). Configuration options tell mod_gzip where to look for pre-compressed files.

mod_gzip_can_negotiate	Yes
mod_gzip_static_suffix	.gz
AddEncoding	gzip	.gz

Newer versions of mod_gzip (starting with version 1.3.26.1a) can pre-compress files for you automatically by adding a single extra configuration option. You’ll need to make sure that Apache has the correct permissions to create and overwrite the gzipped files for this to work.

mod_gzip_update_static	Yes

However, it’s not that simple. Certain versions of Netscape 4 (specifically 4.06 to 4.08) identify themselves as being able to interpret gzipped content (they send a header saying they do), but they cannot correctly decompress it. Most other versions of Netscape 4 have issues with loading compressed JavaScript and CSS in different and exciting ways. We need to detect these agents on the server side and make sure they get served an uncompressed version. This is fairly easy to work around, but Internet Explorer (versions 4 through 6) has some more interesting issues. When loading gzipped JavaScript, Internet Explorer will sometimes incorrectly decompress the resource, or halt compression halfway through, presenting half a file to the client. If you rely on your JavaScript working, you need to avoid sending gzipped content to Internet Explorer. In the cases where Internet Explorer does receive gzipped JavaScript correctly, some older 5.x versions won’t cache the file, regardless of it’s e-tag headers.

Since gzip compression of content is so problematic, we can instead turn our attention to compressing content without changing its format. There are many JavaScript compression scripts available, most of which use a regular expression driven rule set to reduce the size of JavaScript source. There are several things which can be done to make the source smaller - removing comments, collapsing whitespace, shortening privately scoped variable names and removing optional syntax.

Unfortunately, most of these scripts either obtain a fairly low compression rate, or are destructive under certain circumstances (or both). Without understanding the full parse tree, it’s difficult for a compressor to distinguish between a comment and what looks like a comment inside a quoted string. Adding closures to the mix, it’s not easy to find which variables have a private lexical scope using regular expressions, so some variable name shortening techniques will break certain kinds of closure code.

One compressor does avoid this fate - the Dojo Compressor (there’s a ready-to-use version here) works by using Rhino (Mozilla’s JavaScript engine implemented in Java) to build a parse tree, which it then reduces before serializing it to a file. The Dojo Compressor can give pretty good savings for a low cost - a single compression at build time. By building this compression into our build process, it all happens transparently for us. We can add as much whitespace and as many comments as we like to our JavaScript in our development environment, without worrying about bloating our production code.

Compared to JavaScript, CSS is relatively simple to compress. Because of a general lack of quoted strings (typically paths and font names) we can mangle the whitespace using regular expressions. In the cases where we do have quoted strings, we can nearly always collapse a whitespace sequence into a single space (since we don’t tend to find multiple spaces or tabs in URL paths or font names). A simple Perl script should be all we need:

#!/usr/bin/perl

my $data = '';
open F, $ARGV[0] or die "Can't open source file: $!";
$data .= 

在一个讨论web技术的网站vitamin上发现这篇《Serving JavaScript Fast》,读过之后大有收获,茅塞顿开。

先在网上查了查,已经有blog谈到这篇文章(我算是后知后觉了),

有总结要点的《Flickr 的开发者的 Web 应用优化技巧》,也有延伸开来的《接着讲Flickr的八卦》

但似乎没有全文翻译的(这下就好,不会忙了半天发现是无用功)。

之后,就写信问作者可不可以,作者一口答应:“sure - i’d love you to translate it”

,只是要求我翻好之后给他一个链接地址。得到准许,心里就有底了。

先介绍一下作者。Cal Henderson,伦敦人,现居加利福尼亚的旧金山。

PHP,MySQL和Perl专家,现任flickr架构师

(flickr被收购后就在yahoo了),同时也是vitamin的特聘顾问(写些技术性文章)。

既然他是架构师,flickr用的应该就是文中谈到的这些技术,于是参照文章,再对比网站,

种种迹象表明确实如此。

虽然在中国访问flickr速度不敢恭维,加速效果不得而知,

但其用了n多css和javascript资源却似乎从没出过什么问题,

也从侧面印证了这些技术的有效性。

先在网上查了查,已经有blog谈到这篇文章(我算是后知后觉了),

有总结要点的《Flickr 的开发者的 Web 应用优化技巧》

也有延伸开来的《接着讲Flickr的八卦》,但似乎没有全文翻译的(这下就好,不会忙了半天发现是无用功)。

之后,就写信问作者可不可以,作者一口答应:“sure - i’d love you to translate it”,

只是要求我翻好之后给他一个链接地址。得到准许,心里就有底了。

先介绍一下作者。Cal Henderson,伦敦人,现居加利福尼亚的旧金山。

 

PHP,MySQL和Perl专家,现任flickr架构师(flickr被收购后就在yahoo了),

 

同时也是vitamin的特聘顾问(写些技术性文章)。

既然他是架构师,flickr用的应该就是文中谈到的这些技术,于是参照文章,再对比网站,

 

种种迹象表明确实如此。虽然在中国访问flickr速度不敢恭维,加速效果不得而知,

 

但其用了n多css和javascript资源却似乎从没出过什么问题,也从侧面印证了这些技术的有效性。

仔细的看完文章,还有个强烈的感觉:这老兄也太能卖关子了,一句话非分成三句说,

 

摆事实讲道理是够透彻,就是有点太@#$%了…… 算了,

 

他怎么说我怎么翻吧,忠实于原著嘛,要不就成篡改了。

 

经过几天努力,加上同事thincat兄倾力援手(小弟不胜感激啊),

 

终于完工(@_@ 真是苦力活啊,我再也不想干了~)。

全文翻译如下:

让javascript跑得更快

作者:Cal Henderson

下一代web应用让javascript和css得堪大用。我们会告诉你怎样使这些应用又快又灵。

建立了号称“Web 2.0”的应用,也实现了富内容(rich content)和交互,

 

我们期待着css和javascript扮演更加重要的角色。为使应用干净利落,我们需要完善那些渲染页面的文件,

 

优化其大小和形态,以确保提供最好的用户体验——在实践中,

 

这就意味着一种结合:使内容尽可能小、下载尽可能快,同时避免对未改动资源不必要的重新获取。

由于css和js文件的形态,情况有点复杂。跟图片相比,其源代码很有可能频繁改动。

 

而一旦改动,就需要客户端重新下载,使本地缓存无效(保存在其他缓存里的版本也是如此)。

 

在这篇文章里,我们将着重探讨怎样使用户体验最快:包括初始页面的下载,随后页面的下载,

 

以及随着应用渐进、内容变化而进行的资源下载。

我始终坚信这一点:对开发者来说,应该尽可能让事情变得简单。

 

所以我们青睐于那些能让系统自动处理优化难题的方法。

 

只需少许工作量,我们就能建立一举多得的环境:它使开发变得简单,有极佳的终端性能,

 

也不会改变现有的工作方式。

好大一沱

老的思路是,为优化性能,可以把多个css和js文件合并成极少数大文件。

 

跟十个5k的js文件相比,合并成一个50k的文件更好。虽然代码总字节数没变,

 

却避免了多个HTTP请求造成的开销。每个请求都会在客户端和服务器两边有个建立和消除的过程,

 

导致请求和响应header带来开销,还有服务器端更多的进程和线程资源消耗

 

(可能还有为压缩内容耗费的cpu时间)。

(除了HTTP请求,)并发问题也很重要。默认情况下,在使用持久连接(persistent connections)时,

 

ie和firefox在同一域名内只会同时下载两个资源(在HTTP 1.1规格书中第8.1.4节的建议)

 

(htmlor注:可以通过修改注册表等方法改变这一默认配置)。

 

这就意味着,在我们等待下载2个js文件的同时,

 

将无法下载图片资源。也就是说,这段时间内用户在页面上看不到图片。

(虽然合并文件能解决以上两个问题,)可是,这个方法有两个缺点。

 

第一,把所有资源一起打包,将强制用户一次下载完所有资源。

 

如果(不这么做,而是)把大块内容变成多个文件,下载开销就分散到了多个页面,

 

同时缓解了会话中的速度压力(或完全避免了某些开销,这取决于用户选择的路径)。

 

如果为了随后页面下载得更快而让初始页面下载得很慢,

我们将发现更多用户根本不会傻等着再去打开下一个页面。

第二(这个影响更大,一直以来却没怎么被考虑过),在一个文件改动很频繁的环境里,

 

如果采用单文件系统,那么每次改动文件都需要客户端把所有css和js重新下载一遍。

 

假如我们的应用有个100k的合成的js大文件,任何微小的改动都将强制客户端把这100k再消化一遍。

分解之道

(看来合并成大文件不太合适。)替代方案是个折中的办法:把css和js资源分散成多个子文件,

 

按功能划分、保持文件个数尽可能少。这个方案也是有代价的,

 

虽说开发时代码分散成逻辑块(logical chunks)能提高效率,可在下载时为提高性能还得合并文件。

 

不过,只要给build系统(把开发代码变成产品代码的工具集,是为部署准备的)加点东西,就没什么问题了。

对于有着不同开发和产品环境的应用来说,用些简单的技术可以让代码更好管理。

 

在开发环境下,为使条理清晰,代码可以分散为多个逻辑部分(logical components)。

 

可以在Smarty(一种php模板语言)里建立一个简单的函数来管理javascript的下载:

SMARTY:
{insert_js files="foo.js,bar.js,baz.js"}

PHP:
function smarty_insert_js($args){
  foreach (explode(',', $args['files']) as $file){
    echo "<script type=\"text/javascript\" SOURCE=\"/javascript/$file\"></script>\n";
  }
}

OUTPUT:
<script type="text/javascript" SOURCE="/javascript/foo.js"></script>
<script type="text/javascript" SOURCE="/javascript/bar.js"></script>
<script type="text/javascript" SOURCE="/javascript/baz.js"></script>

(htmlor注:wordpress中会把“src”替换成不知所谓的字符,因此这里只有写成“SOURCE”,

 

使用代码时请注意替换,下同)

就这么简单。然后我们就命令build过程(build process)去把确定的文件合并起来。

 

这个例子里,合并的是foo.js和bar.js,因为它们几乎总是一起下载。

 

我们能让应用配置记住这一点,并修改模板函数去使用它。(代码如下:)

SMARTY:
{insert_js files="foo.js,bar.js,baz.js"}

PHP:
# 源文件映射图。在build过程合并文件之后用这个图找到js的源文件。

$GLOBALS['config']['js_source_map'] = array(
  'foo.js'	=> 'foobar.js',
  'bar.js'	=> 'foobar.js',
  'baz.js'	=> 'baz.js',
);

function smarty_insert_js($args){
  if ($GLOBALS['config']['is_dev_site']){
    $files = explode(',', $args['files']);
  }else{
    $files = array();
    foreach (explode(',', $args['files']) as $file){
      $files[$GLOBALS['config']['js_source_map'][$file]]++;
    }
    $files = array_keys($files);
  }

  foreach ($files as $file){
    echo "<script type=\"text/javascript\" SOURCE=\"/javascript/$file\"></script>\n";
  }
}

OUTPUT:
<script type="text/javascript" SOURCE="/javascript/foobar.js"></script>
<script type="text/javascript" SOURCE="/javascript/baz.js"></script>

模板里的源代码没必要为了分别适应开发和产品阶段而改动,它帮助我们在开发时保持文件分散,

 

发布成产品时把文件合并。想更进一步的话,可以把合并过程(merge process)写在php里,

 

然后使用同一个(合并文件的)配置去执行。这样就只有一个配置文件,避免了同步问题。

 

为了做的更加完美,我们还可以分析css和js文件在页面中同时出现的几率,

 

以此决定合并哪些文件最合理(几乎总是同时出现的文件是合并的首选)。

对css来说,可以先建立一个主从关系的模型,它很有用。一个主样式表控制应用的所有样式表,

 

多个子样式表控制不同的应用区域。采用这个方法,大多数页面只需下载两个css文件,

 

而其中一个(指主样式表)在页面第一次请求时就会缓存。

对没有太多css和js资源的应用来说,这个方法在第一次请求时可能比单个大文件慢,

 

但如果保持文件数量很少的话,你会发现其实它更快,因为每个页面的数据量更小。

 

让人头疼的下载花销被分散到不同的应用区域,因此并发下载数保持在一个最小值,

 

同时也使得页面的平均下载数据量很小。

压缩

谈到资源压缩,大多数人马上会想到mod_gzip(但要当心,mod_gzip实际上是个魔鬼,至少能让人做恶梦)。

 

它的原理很简单:浏览器请求资源时,会发送一个header表明自己能接受的内容编码。就像这样:

Accept-Encoding: gzip,deflate

服务器遇到这样的header请求时,就用gzip或deflate压缩内容发往客户端,然后客户端解压缩。

 

这过程减少了数据传输量,同时消耗了客户端和服务器的cpu时间。也算差强人意。

 

但是,mod_gzip的工作方式是这样的:先在磁盘上创建一个临时文件,然后发送(给客户端),

 

最后删除这个文件。在高容量的系统中,由于磁盘io问题,很快就会达到极限。

 

要避免这种情况,可以改用mod_deflate(apache 2才支持)。

 

它采用更合理的方式:在内存里做压缩。对于apache 1的用户来说,可以建立一块ram磁盘,

 

让mod_gzip在它上面写临时文件。虽然没有纯内存方式快,但也不会比往磁盘上写文件慢。

话虽如此,其实还是有办法完全避免压缩开销的,那就是预压缩相关静态资源,

 

下载时由mod_gzip提供合适的压缩版本。如果把压缩添加在build过程,它就很透明了。

 

需要压缩的文件通常很少(用不着压缩图片,因为并不能减小更多体积),

 

只有css和js文件(和其他未压缩的静态内容)。

配置选项会告诉mod_gzip去哪里找到预压缩过的文件。

mod_gzip_can_negotiate	Yes
mod_gzip_static_suffix	.gz
AddEncoding	gzip	.gz

新一点的mod_gzip版本(从1.3.26.1a开始)添加一个额外的配置选项后,

 

就能自动预压缩文件。不过在此之前,必须确认apache有正确的权限去创建和覆盖压缩文件。

mod_gzip_update_static	Yes

可惜,事情没那么简单。某些Netscape 4的版本(尤其是4.06-4.08)认为自己能够解释压缩内容

 

(它们发送一个header这么说来着),但其实它们不能正确的解压缩。

 

大多数其他版本的Netscape 4在下载压缩内容时也有各种各样的问题。

 

所以要在服务器端探测代理类型,(如果是Netscape 4,就要)让它们得到未压缩的版本。

 

这还算简单的。ie(版本4-6)有些更有意思的问题:当下载压缩的javascript时,

 

有时候ie会不正确的解压缩文件,或者解压缩到一半中断,然后把这半个文件显示在客户端。

 

如果你的应用对javascript的依赖比较大(htmlor注:比如ajax应用),那么就得避免发送压缩文件给ie。

 

在某些情况下,一些更老的5.x版本的ie倒是能正确的收到压缩的javascript,

 

可它们会忽略这个文件的etag header,不缓存它。

 

(thincat友情提示:尽管压缩存在一些浏览器不兼容的现象,

 

由于这些不能很好的支持压缩的浏览器数量现在已经非常少了,

 

我认为这种由于浏览器导致的压缩不正常的情况可以忽略不计。

 

这些过时的浏览器还能不能在现在流行的windows或unix环境下面安装都存在不小的问题)

既然gzip压缩有这么多问题,我们不妨把注意力转到另一边:不改变文件格式的压缩。

 

现在有很多这样的javascript压缩脚本可用,大多数都用一个正则表达式驱动的语句集来减小源代码的体积。

 

它们做的不外乎几件事:去掉注释,压缩空格,缩短私有变量名和去掉可省略的语法。

不幸的是,大多数脚本效果并不理想,要么压缩率相当低,

 

要么某种情形下会把代码搞得一团糟(或者两者兼而有之)。

 

由于对解析树的理解不完整,压缩器很难区分一句注释和一句看似注释的引用字符串。

 

因为闭合结构的混合使用,要用正则表达式发现哪些变量是私有的并不容易,

 

因此一些缩短变量名的技术会打乱某些闭合代码。

还好有个压缩器能避免这些问题:dojo压缩器(现成的版本在这里)。

 

它使用rhino(mozilla的javascript引擎,是用java实现的)建立一个解析树,

 

然后将其提交给文件。它能很好的减小代码体积,

 

仅用很小的成本:因为只在build时压缩一次。由于压缩是在build过程中实现的,

 

所以一清二楚。(既然压缩没有问题了,)我们可以在源代码里随心所欲的添加空格和注释,

 

而不必担心影响到产品代码。

与javascript相比,css文件的压缩相对简单一些。由于css语法里不会有太多引用字符串

 

(通常是url路径跟字体名),我们可以用正则表达式大刀阔斧的干掉空格

 

(htmlor注:这句翻的最爽,哈哈)。如果确实有引用字符串的话,我们总可以把一串空格合成一个

 

(因为不需要在url路径和字体名里查找多个空格和tab)。这样的话,一个简单的perl脚本就够了:

___FCKpd___5

然后,就可以把单个的css文件传给脚本去压缩了。命令如下:

perl compress.pl site.source.css > site.compress.css

做完这些简单的纯文本优化工作后,我们就能减少数据传输量多达50%了(这个量取决于你的代码格式,可能更多)。

 

这带来了更快的用户体验。不过我们真正想做的是,尽可能避免用户请求的发生——除非确实有必要。

 

这下HTTP缓存知识派上用场了。

 

缓存真是个好东西

看来我们正从错误的方向入手解决问题。(现在的问题是,)这些可能的缓存策略导致了一件事情反复发生,

 

那就是:客户端向服务器查询本地缓存是否最新。假如服务器在改动文件的时候通知客户端,

 

客户端不就知道它的缓存是最新的了(直到接到下一次通知)?可惜天公不做美——(事实)

 

是客户端向服务器发出请求。

其实,也不尽然。在获取js或css文件之前,客户端会用<script>或<link>标记向服务器发送一个请求,

 

说明哪个页面要加载这些文件。这时候就可以用服务器的响应来通知客户端这些文件有了改动。

 

有点含糊,说得再详细点就是:如果改变css和js文件内容的同时,也改变它们的文件名,

 

就可以告诉客户端对url全都永久缓存——因为每个url都是唯一的。

假如能确定一个资源永不更改,我们就可以发出一些霸气十足的缓存header(htmlor注:这句也很有气势吧)。

 

在php里,两行就好:

<?php
header("Expires: ".gmdate("D, d M Y H:i:s", time()+315360000)." GMT");
header("Cache-Control: max-age=315360000");
?>

我们告诉浏览器这个内容在10年后(10年大概会有315,360,000秒,或多或少)过期,

 

浏览器将会保留它10年。当然,很有可能不用php输出css和js文件(因此就不能发出header),

 

这种情况将在稍后说明。

人力有时而穷

当文件内容更改时,手动去改文件名是很危险的。假如你改了文件名,模板却没有指向它?

 

假如你改了一些模板另一些却没改?假如你改了模板却没改文件名?还有最糟的,

 

假如你改动了文件却忘了改名或者忘了改变对它的引用?最好的结果,是用户看到老的而看不到新的内容。

 

最坏的结果,是找不到文件,网站没法运转了。听起来这(指改动文件内容时修改url)似乎是个馊主意。

幸运的是,计算机做这类事情——当某种变化发生,需要相当准确地完成的、

 

重复重复再重复的(htmlor注:番茄鸡蛋伺候~)、枯燥乏味的工作——总是十分在行。

这个过程(改变文件的url)没那么痛苦,因为我们根本不需要改文件名。

 

资源的url和磁盘上文件的位置也没必要保持一致。

 

使用apache的mod_rewrite模块,可以建立简单的规则,让确定的url重定向到确定的文件。

RewriteEngine on
RewriteRule ^/(.*.)v[0-9.]+.(css|js|gif|png|jpg)$	/$1$2	[L]

这条规则匹配任何带有指定扩展名同时含有“版本”信息(version nugget)的url,

 

它会把这些url重定向到一个不含版本信息的路径。如下所示:

URL			   Path
/images/foo.v2.gif	-> /images/foo.gif
/css/main.v1.27.css	-> /css/main.css
/javascript/md5.v6.js	-> /javascript/md5.js

使用这条规则,就可以做到不改变文件路径而更改url(因为版本号变了)。

 

由于url变了,浏览器就认为它是另一个资源(会重新下载)。想更进一步的话,

 

可以把我们之前说的脚本编组函数结合起来,根据需要生成一个带有版本号的<script>标记列表。

说到这里,你可能会问我,为什么不在url结尾加一个查询字符串(query string)呢

 

(如/css/main.css?v=4)?根据HTTP缓存规格书所说,用户代理对含有查询字符串的url永不缓存。

 

虽然ie跟firefox忽略了这点,opera和safari却没有——为了确保所有浏览器都缓存你的资源,

 

还是不要在url里用查询字符串的好。

现在不移动文件就能更改url了,如果能让url自动更新就更好了。

 

在小型的产品环境下(如果有大型的产品环境,就是开发环境了),

 

使用模板功能可以很轻易的实现这点。这里用的是smarty,用其他模板引擎也行。

SMARTY:
<link xhref="{version xsrc='/css/group.css'}" rel="stylesheet" type="text/css" />

PHP:
function smarty_version($args){
  $stat = stat($GLOBALS['config']['site_root'].$args['src']);
  $version = $stat['mtime'];

  echo preg_replace('!.([a-z]+?)$!', ".v$version.$1", $args['src']);
}

OUTPUT:
<link xhref="/css/group.v1234567890.css" mce_href="/css/group.v1234567890.css" 
rel="stylesheet" type="text/css" />

对每个链接到的资源文件,我们得到它在磁盘上的路径,检查它的mtime(文件最后修改的日期和时间),

然后把这个时间当作版本号插入到url中。对于低流量的站点(它们的stat操作开销不大)或者开发环境来说,

 

这个方案不错,但对于高容量的环境就不适用了——因为每次stat操作都要磁盘读取(导致服务器负载升高)。

解决方案相当简单。在大型系统中每个资源都已经有了一个版本号,就是版本控制的修订号

 

(你们应该使用了版本控制,对吧?)。当我们建立站点准备部署的时候,可以轻易的查到每个文件的修订号,

 

写在一个静态配置文件里。

<?php
$GLOBALS['config']['resource_versions'] = array(
  '/images/foo.gif'    => '2.1',
  '/css/main.css'      => '1.27',
  '/javascript/md5.js' => '6.1.4',
);
?>

当我们发布产品时,可以修改模板函数来使用版本号。

<?php
function smarty_version($args){
  if ($GLOBALS['config']['is_dev_site']){
    $stat = stat($GLOBALS['config']['site_root'].$args['src']);
    $version = $stat['mtime'];
  }else{
    $version = $GLOBALS['config']['resource_versions'][$args['src']];
  }

  echo preg_replace('!.([a-z]+?)$!', ".v$version.$1", $args['src']);
}
?>

就这样,不需要改文件名,也不需要记住改了哪些文件——当文件有新版本发布时它的url就会

 

自动更新——有意思吧?我们就快搞定了。

只欠东风

之前谈到为静态文件发送超长周期(very-long-period)的缓存header时曾说过,

 

如果不用php输出,就不能轻易的发送缓存header。很显然,

 

有两个办法可以解决:用php输出,或者让apache来做。

php出马,手到擒来。我们要做的仅仅是改变rewrite规则,把静态文件指向php脚本,

 

用php在输出文件内容之前发送header。

SMARTY:
{insert_js files="foo.js,bar.js,baz.js"}

PHP:
function smarty_insert_js($args){
  foreach (explode(',', $args['files']) as $file){

   echo "<script type=\"text/javascript\" src=\"/javascript/$file\"></script>\n";
  }
}

OUTPUT:
<script type="text/javascript" src="/javascript/foo.js"></script>
<script type="text/javascript" src="/javascript/bar.js"></script>
<script type="text/javascript" src="/javascript/baz.js"></script>

这个方案有效,但并不出色。(因为)跟apache相比,php需要更多内存和执行时间。

 

另外,我们还得小心防止可能由path参数传递伪造值引起的exploits。

 

为避免这些问题,应该用apache直接发送header。

 

rewrite规则语句允许当规则匹配时设置环境变量(environment variable),

 

当给定的环境变量设置后,Header命令就可以添加header。

 

 

结合以下两条语句,我们就把rewrite规则和header设置绑定在了一起:

SMARTY:
{insert_js files="foo.js,bar.js,baz.js"}

PHP:
# map of where we can find .js source files after the build process
# has merged as necessary

$GLOBALS['config']['js_source_map'] = array(
  'foo.js'	=> 'foobar.js',
  'bar.js'	=> 'foobar.js',
  'baz.js'	=> 'baz.js',
);

function smarty_insert_js($args){

  if ($GLOBALS['config']['is_dev_site']){

    $files = explode(',', $args['files']);
  }else{

    $files = array();

    foreach (explode(',', $args['files']) as $file){

      $files[$GLOBALS['config']['js_source_map'][$file]]++;
    }

    $files = array_keys($files);
  }

  foreach ($files as $file){

   echo "<script type=\"text/javascript\" src=\"/javascript/$file\"></script>\n";
  }
}

OUTPUT:
<script type="text/javascript" src="/javascript/foobar.js"></script>
<script type="text/javascript" src="/javascript/baz.js"></script>

考虑到apache的执行顺序,应该把rewrite规则加在主配置文件(httpd.conf)

 

而不是目录配置文件(.htaccess)中。否则在环境变量设置之前,header行会先执行(就那没意义了)。

 

至于header行,则可以放在两文件任何一个当中,没什么区别。

眼观六路

(htmlor注:多谢tchaikov告知“skinning rabbits”的含义,但我不想翻的太正式,

 

眼下的这个应该不算太离谱吧。)

通过结合使用以上技术,我们可以建立一个灵活的开发环境和一个快速又高性能的产品环境。

 

当然,这离终极目标“速度”还有一段距离。有许多更深层的技术(比如分离伺服静态内容,

 

用多域名提升并发量等)值得我们关注,包括与我们谈到的方法(建立apache过滤器,修改资源url,

 

加上版本信息)殊途同归的其他路子。你可以留下评论,告诉我们那些你正在使用的卓有成效的技术和方法。

(完)

The next generation of web apps make heavy use of JavaScript and CSS. We’ll show you how to make those apps responsive and quick.

With our so-called "Web 2.0" applications and their rich content and interaction, we expect our applications to increasingly make use of CSS and JavaScript. To make sure these applications are nice and snappy to use, we need to optimize the size and nature of content required to render the page, making sure we’re delivering the optimum experience. In practice, this means a combination of making our content as small and fast to download as possible, while avoiding unnecessarily refetching unmodified resources.

This is complicated a little by the nature of CSS and JavaScript resources. In contrast to image assets, CSS and JavaScript source code is very likely to change many times as time goes by. When these resources change, we need our clients to download them all over again, invalidating the version in their local cache (and any versions stored in other caches along the way). In this article, we’ll look at ways we can make the whole experience as fast as possible for our users - the initial page load, subsequent page loads and ongoing resource loading as the application evolves and content changes.

I believe strongly in making things as simple as possible for developers, so we’ll also be looking at ways we can set up our systems to automatically take care of these optimization issues for us. With a little up front work, we can get the best of both worlds - an environment that makes development easy with great end-user performance - all without changing the way we work.

Monolith

The old school of thought was that we could achieve optimal performance by combining multiple CSS and JavaScript files into fewer, larger blocks. Rather than having ten 5k JavaScript files, we combine them into a single 50k file. While the total size of the code is still the same, we avoid having the overhead associated with multiple HTTP requests. Each request has a setup and teardown phase on both the client and server, incurs request and response header size overhead, and resource overhead on the server side in the form of more processes or threads (and perhaps more CPU time for on-the-fly gzipped content).

The parellization aspect is also important. By default, both Internet Explorer and Mozilla/Firefox will only download two resources from a single domain at once when using persistent connections (as suggested in the HTTP 1.1 spec, section 8.1.4). This means that while we’re waiting to download those JavaScript files, 2 at a time, we’re not loading image assets - the page our users see during the loading phase will be missing its images.

However, there are a couple of downsides to this approach. By bundling all of our resources together, we force the user to download everything up front. By chunking content into multiple files we can spread out the cost of loading across several pages, amortizing the speed hit across a session (or avoiding some of the cost completely, depending on the path the user chooses). If we make the first page slow to speed up subsequent pages, we might find that we have more users who never wait around to request a second page.

The big downside to the single file approach has not often, historically, been considered. In an environment where we will have to often change our resources, any changes to a single-file system will require the client to re-download a copy of the entire CSS or JavaScript working set. If our application has a single monolithic 100k JavaScript source file, any tiny change to our code will force all clients to suck down the 100k all over again.

A splintered approach

The alternative approach lies somewhere in the middle - we split our CSS and JavaScript resources into multiple sub-files, while at the same time keeping that number functionally low. This compromise comes at a cost - we need to be able to develop applications with our code split out into logical chunks to increase development efficiency, while delivering merged files for performance. With a few additions to our build system (the set of tools which turn your development code into production code, ready for deployment), this needn’t be a compromise we have to make.

For an application environment with distinct development and production environments, you can use a few simple techniques to keep your code manageable. In your development environment, code can be split into many logical components to make separation clear. In Smarty (A PHP templating language) we can create a simple function to manage the loading of our JavaScript:

___FCKpd___16

So far, so easy. But then we instruct our build process to merge certain files together into single resources. In our example, imagine we merged foo.js and bar.js into foobar.js, since they are nearly always loaded together. We can then record this fact in our application configuration and modify our template function to use this information.

___FCKpd___17

The source code in our templates doesn’t need to change between development and production, but allows us to keep files separated while developing and merged in production. For bonus points, we can write our merging process in PHP and use the same configuration block to perform the merge process, allowing us to keep a single configuration file and avoid having to keep anything in sync. For super-bonus points, we could analyze the occurrence of scripts and style sheets together on pages we serve, to determine which files would be best to merge (files that nearly always appear together are good candidates for merging).

For CSS, a useful model to start from is that of a master and subsection relationship. A single master style sheet controls style across your entire application, while multiple sub-sheets control various distinct feature areas. In this way, most pages will load only two sheets, one of which is cached the first time any page is requested (the master sheet).

For small CSS and JavaScript resource sets, this approach may be slower for the first request than a single large resource, but if you keep the number of components low then you’ll probably find it’s actually faster, since the data size per page is much lower. The painful loading costs are spread out around different application areas, so the number of parallel loads is kept to a minimum while also keeping the resources-per-page size low.

Compression

When talk about asset compression, most people think immediately of mod_gzip. Beware, however - mod_gzip is actually evil, or at the least, a resource hogging nightmare. The idea behind it is simple - browsers request resources and send along a header to show what kind of content encodings they accept. It looks something like this:

Accept-Encoding: gzip,deflate

When a server encounters this header, it can then gzip or deflate (compress) the content it’s sending to the client, where the client will then decompress it. This burns CPU time on both the client and server, while reducing the amount of data transferred. All well and good. The way mod_gzip works, however, is to create a temporary file on disk in which to compress the source data, serve that file out, then delete it. For high volume systems, you very quickly become bound by disk IO. We can avoid this by using mod_deflate instead (Apache 2 only), which does all the compression in memory - sensible. For Apache 1 users, you can instead create a RAM disk and have mod_gzip writes its temporary files there - not quite as fast as pure in-memory compression, but not nearly as slow as writing to disk.

Even so, we can avoid the compression overhead completely by pre-compressing the relevant static resources and using mod_gzip to serve people the compressed version where appropriate. If we add this compression into our build process, it all happens transparently to us. The number of files that need compressing is typically quite low - we don’t compress images since we don’t gain much, if any, size benefit (since they’re already compressed) so we only need to compress our JavaScript and CSS (and any other uncompressed static content). Configuration options tell mod_gzip where to look for pre-compressed files.

mod_gzip_can_negotiate	Yes
mod_gzip_static_suffix	.gz
AddEncoding	gzip	.gz

Newer versions of mod_gzip (starting with version 1.3.26.1a) can pre-compress files for you automatically by adding a single extra configuration option. You’ll need to make sure that Apache has the correct permissions to create and overwrite the gzipped files for this to work.

mod_gzip_update_static	Yes

However, it’s not that simple. Certain versions of Netscape 4 (specifically 4.06 to 4.08) identify themselves as being able to interpret gzipped content (they send a header saying they do), but they cannot correctly decompress it. Most other versions of Netscape 4 have issues with loading compressed JavaScript and CSS in different and exciting ways. We need to detect these agents on the server side and make sure they get served an uncompressed version. This is fairly easy to work around, but Internet Explorer (versions 4 through 6) has some more interesting issues. When loading gzipped JavaScript, Internet Explorer will sometimes incorrectly decompress the resource, or halt compression halfway through, presenting half a file to the client. If you rely on your JavaScript working, you need to avoid sending gzipped content to Internet Explorer. In the cases where Internet Explorer does receive gzipped JavaScript correctly, some older 5.x versions won’t cache the file, regardless of it’s e-tag headers.

Since gzip compression of content is so problematic, we can instead turn our attention to compressing content without changing its format. There are many JavaScript compression scripts available, most of which use a regular expression driven rule set to reduce the size of JavaScript source. There are several things which can be done to make the source smaller - removing comments, collapsing whitespace, shortening privately scoped variable names and removing optional syntax.

Unfortunately, most of these scripts either obtain a fairly low compression rate, or are destructive under certain circumstances (or both). Without understanding the full parse tree, it’s difficult for a compressor to distinguish between a comment and what looks like a comment inside a quoted string. Adding closures to the mix, it’s not easy to find which variables have a private lexical scope using regular expressions, so some variable name shortening techniques will break certain kinds of closure code.

One compressor does avoid this fate - the Dojo Compressor (there’s a ready-to-use version here) works by using Rhino (Mozilla’s JavaScript engine implemented in Java) to build a parse tree, which it then reduces before serializing it to a file. The Dojo Compressor can give pretty good savings for a low cost - a single compression at build time. By building this compression into our build process, it all happens transparently for us. We can add as much whitespace and as many comments as we like to our JavaScript in our development environment, without worrying about bloating our production code.

Compared to JavaScript, CSS is relatively simple to compress. Because of a general lack of quoted strings (typically paths and font names) we can mangle the whitespace using regular expressions. In the cases where we do have quoted strings, we can nearly always collapse a whitespace sequence into a single space (since we don’t tend to find multiple spaces or tabs in URL paths or font names). A simple Perl script should be all we need:

___FCKpd___21

We can then feed individual CSS files through the script to compress them like so:

perl compress.pl site.source.css > site.compress.css

With these simple plaintext optimizations we can reduce the amount of data sent over the wire by as much as 50% (depending upon your coding style - it might be much less), which can translate to a much faster experience for our users. But what we’d really like to do is avoid users having to even request files unless completely necessary - and that’s where an intimate knowledge of HTTP caching comes in handy.

Caching is your friend

When a user agent requests a resource from a server for the first time, it caches the response to avoid making the same request in the future. How long it stores this response for is influenced by two factors - the agent configuration and any cache control response headers from the server. All browsers have subtly different configuration options and behaviors, but most will cache a given resource for at least the length of a session, unless explicitly told otherwise.

It’s quite likely you already send out anti-caching headers for dynamic content pages to avoid the browser caching pages which constantly change. In PHP, you can achieve this with a pair of function calls:

<?php
  header("Cache-Control: private");
  header("Cache-Control: no-cache", false);
?>

Sounds too easy? It is - some agents will ignore this header under certain circumstances. To really convince a browser not to cache a document, you’ll need to be a little more forceful:

<?php
# 'Expires' in the past
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT");

# Always modified
header("Last-Modified: ".gmdate("D, d M Y H:i:s")." GMT");

# HTTP/1.1
header("Cache-Control: no-store, no-cache, must-revalidate");
header("Cache-Control: post-check=0, pre-check=0", false);

# HTTP/1.0
header("Pragma: no-cache");
?>

This is fine for content we don’t want to be cached, but for content that doesn’t change with every request we want to encourage the browser to cache it aggressively. The “If-Modified-Since” request header allows us to get part of the way there. If a client sends an “If-Modified-Since” header with its request, Apache (or your web server of choice) can respond with status code 304 (”Not Modified”), telling the browser that its cached copy of the file is already up to date. With this mechanism, we can avoid sending the contents of a file to the browser, but we still incur the overhead of an HTTP request. Hmmm.

Similar to the if-modified-since mechanism are entity tags. Under Apache, each response for a static resource is given an “ETag” header containing a checksum generated from the file’s modified-time, size and inode number. A browser can then perform a HEAD request to check the e-tag for a resource without downloading it. E-tags suffer from the same problem as the if-modified-since mechanism - the client still needs to perform an HTTP request to determine the validity of the locally cached copy.

In addition, you need to be careful with if-modified-since and e-tags if you serve content from multiple servers. With two load-balanced web servers, a single resource could be requested from either server by a single agent - and could be requested from each at different times. This is great - it’s why we load balance. However, if the two servers generate different e-tags or modified dates for the same files, then browsers won’t be able to properly cache content. By default, e-tags are generated using the inode number of the file, which will vary from server to server. You can turn this off using a single Apache configuration option:

FileETag MTime Size

With this option, Apache will use only the modification time and file size to determine the e-tag. This, unfortunately, leads us to the other problem with e-tags, which can affect if-modified-since too (though not nearly as badly). Since the e-tag relies on the modified time of the file, we need those times to be in sync. If we’re pushing files to multiple web servers, there’s always a chance that the time at which the files are pushed are subtly different by a second or two. In this case, the e-tags generated by two servers will still be different. We could change the configuration to generate e-tags only from the file size, but this means that we’ll generate the same e-tag if we change a file’s contents without changing its size. Not ideal.

Caching is your best friend

The problem here is that we are approaching the issue from the wrong direction. These possible caching strategies all revolve around the client asking the server if its cached copy is fresh. If we could notify the client when we change a file, it would know that its own cached copy was fresh, until we told it otherwise. But the web doesn’t work that way - the client makes requests to the server.

But that’s not quite true - before fetching any JavaScript or CSS files, the client makes a request to the server for the page which will be loading those files via <script> or <link> tags. We can use the response from the server to notify the client of any changes in those resources. This is all a little cryptic, so let’s spell it out - if we change the filenames of JavaScript and CSS files when we change their contents, we can tell the client to cache every URL forever, since the content of any given URL will never change.

If we are sure that a given resource will never change, then we can send out some seriously aggressive caching headers. In PHP, we just need a couple of lines:

<?php
header("Expires: ".gmdate("D, d M Y H:i:s", time()+315360000)." GMT");
header("Cache-Control: max-age=315360000");
?>

Here we tell the browser that the content will expire in 10 years (there are 315,360,000 seconds in 10 years, more or less) and that it can keep it around for 10 years. Of course, we’re probably not serving our JavaScript and CSS via PHP - we’ll address that in a few moments.

Mistakes abound

Manually changing the filenames of resources when the contents are modified is a dangerous task. What happens if you rename the file, but not the templates pointing to it? What happens if you change some templates but not others? What happens if you change the templates but don’t rename the file? Most likely of all, what happens if you modify a resource but forget to rename it or change any references to it. In the best of these cases, users will not see the new content and be stuck with the old versions. In the worst case, no valid resource is found and your site stops working. This sounds like a dumb idea.

Luckily computers are really good at this sort of thing - dull repetitive tasks which need to be done exactly right, over and over again, when some kind of change occurs.

The first step in making this process as painless as possible is to realize that we don’t need to rename files at all. URLs we serve content from and where the content is located on disk don’t have to have anything to do with each other. Using Apache’s mod_rewrite we can create a simple rule to redirect certain URLs to certain files.

RewriteEngine on
RewriteRule ^/(.*\.)v[0-9.]+\.(css|js|gif|png|jpg)$	/$1$2	[L]

This rule matches any URL with one of the specified extensions which also contains a ‘version’ nugget. The rule then rewrites these URLs to a path without the version nugget. Some examples:

URL			   Path
/images/foo.v2.gif	-> /images/foo.gif
/css/main.v1.27.css	-> /css/main.css
/javascript/md5.v6.js	-> /javascript/md5.js

With this rule in-place, we can change the URL (by changing the version number) without changing where the file lives on disk. Because the URL has changed, the browser treats it as a different resource. For bonus points, you can combine this with the script grouping function from earlier to produce a list of versioned <script> tags as needed.

At this point, you might ask why we don’t just add a query string to the end of the resource - /css/main.css?v=4. According the letter of the HTTP caching specification, user agents should never cache URLs with query strings. While Internet Explorer and Firefox ignore this, Opera and Safari don’t - to make sure all user agents can cache your resources, we need to keep query strings out of their URLs.

Now that we can change our URLs without moving the file, it would be nice to be able to have the URLs updated automatically. In a small production environment (or a development environment, for people with large production environments), we can do this really easily using a template function. This example is for Smarty, but applies equally well to other templating engines.

SMARTY:
<link href="{version src='/css/group.css'}" rel="stylesheet" type="text/css" />

PHP:
function smarty_version($args){

  $stat = stat($GLOBALS['config']['site_root'].$args['src']);
  $version = $stat['mtime'];

  echo preg_replace('!\.([a-z]+?)$!', ".v$version.\$1", $args['src']);
}

OUTPUT:
<link href="/css/group.v1234567890.css" rel="stylesheet" type="text/css" />

For each linked resource, we determine the file’s location on disk, check its mtime (the date and time the file was last modified on disk) and insert that into the URL as the version number. This works great for low traffic sites (where stat operations are cheap) and for development environments, but it doesn’t scale well to high volume deployments - each call to stat requires a disk read.

The solution is fairly simple. In a large system we already have a version number for each resource, in the form of the source control revision number (you’re already using source control, right?). At the point when we go to build our site for deployment, we simply check the revision numbers of all of our resource files and write them to a static configuration file.

<?php
$GLOBALS['config']['resource_versions'] = array(

  '/images/foo.gif'    => '2.1',
  '/css/main.css'      => '1.27',
  '/javascript/md5.js' => '6.1.4',
);
?>

We can then modify our templating function to use these version numbers when we’re operating in production.

<?php
function smarty_version($args){

  if ($GLOBALS['config']['is_dev_site']){

    $stat = stat($GLOBALS['config']['site_root'].$args['src']);
    $version = $stat['mtime'];
  }else{
    $version = $GLOBALS['config']['resource_versions'][$args['src']];
  }

  echo preg_replace('!\.([a-z]+?)$!', ".v$version.\$1", $args['src']);
}
?>

In this way, we don’t need to rename any files, or even remember when we modify resources - the URL will be automatically changed everywhere whenever we push out a new revision - lovely. We’re almost where we want to be.

Bringing it all together

When we talked about sending very-long-period cache headers with our static resources earlier, we noted that since this content isn’t usually served through PHP, we can’t easily add the cache headers. We have a couple of obvious choices for dealing with this; inserting PHP into the process or letting Apache do the work.

Getting PHP to do our work for us is fairly simple. All we need to do is change the rewrite rule for the static files to be routed through a PHP script, then have the PHP script output headers before outputting the content of the requested resource.

Apache:
RewriteRule ^/(.*\.)v[0-9.]+\.(css|js|gif|png|jpg)$  /redir.php?path=$1$2  [L]

PHP:
header("Expires: ".gmdate("D, d M Y H:i:s", time()+315360000)." GMT");
header("Cache-Control: max-age=315360000");

# ignore paths with a '..'
if (preg_match('!\.\.!', 

在一个讨论web技术的网站vitamin上发现这篇《Serving JavaScript Fast》,读过之后大有收获,茅塞顿开。

先在网上查了查,已经有blog谈到这篇文章(我算是后知后觉了),

有总结要点的《Flickr 的开发者的 Web 应用优化技巧》,也有延伸开来的《接着讲Flickr的八卦》

但似乎没有全文翻译的(这下就好,不会忙了半天发现是无用功)。

之后,就写信问作者可不可以,作者一口答应:“sure - i’d love you to translate it”

,只是要求我翻好之后给他一个链接地址。得到准许,心里就有底了。

先介绍一下作者。Cal Henderson,伦敦人,现居加利福尼亚的旧金山。

PHP,MySQL和Perl专家,现任flickr架构师

(flickr被收购后就在yahoo了),同时也是vitamin的特聘顾问(写些技术性文章)。

既然他是架构师,flickr用的应该就是文中谈到的这些技术,于是参照文章,再对比网站,

种种迹象表明确实如此。

虽然在中国访问flickr速度不敢恭维,加速效果不得而知,

但其用了n多css和javascript资源却似乎从没出过什么问题,

也从侧面印证了这些技术的有效性。

先在网上查了查,已经有blog谈到这篇文章(我算是后知后觉了),

有总结要点的《Flickr 的开发者的 Web 应用优化技巧》

也有延伸开来的《接着讲Flickr的八卦》,但似乎没有全文翻译的(这下就好,不会忙了半天发现是无用功)。

之后,就写信问作者可不可以,作者一口答应:“sure - i’d love you to translate it”,

只是要求我翻好之后给他一个链接地址。得到准许,心里就有底了。

先介绍一下作者。Cal Henderson,伦敦人,现居加利福尼亚的旧金山。

 

PHP,MySQL和Perl专家,现任flickr架构师(flickr被收购后就在yahoo了),

 

同时也是vitamin的特聘顾问(写些技术性文章)。

既然他是架构师,flickr用的应该就是文中谈到的这些技术,于是参照文章,再对比网站,

 

种种迹象表明确实如此。虽然在中国访问flickr速度不敢恭维,加速效果不得而知,

 

但其用了n多css和javascript资源却似乎从没出过什么问题,也从侧面印证了这些技术的有效性。

仔细的看完文章,还有个强烈的感觉:这老兄也太能卖关子了,一句话非分成三句说,

 

摆事实讲道理是够透彻,就是有点太@#$%了…… 算了,

 

他怎么说我怎么翻吧,忠实于原著嘛,要不就成篡改了。

 

经过几天努力,加上同事thincat兄倾力援手(小弟不胜感激啊),

 

终于完工(@_@ 真是苦力活啊,我再也不想干了~)。

全文翻译如下:

让javascript跑得更快

作者:Cal Henderson

下一代web应用让javascript和css得堪大用。我们会告诉你怎样使这些应用又快又灵。

建立了号称“Web 2.0”的应用,也实现了富内容(rich content)和交互,

 

我们期待着css和javascript扮演更加重要的角色。为使应用干净利落,我们需要完善那些渲染页面的文件,

 

优化其大小和形态,以确保提供最好的用户体验——在实践中,

 

这就意味着一种结合:使内容尽可能小、下载尽可能快,同时避免对未改动资源不必要的重新获取。

由于css和js文件的形态,情况有点复杂。跟图片相比,其源代码很有可能频繁改动。

 

而一旦改动,就需要客户端重新下载,使本地缓存无效(保存在其他缓存里的版本也是如此)。

 

在这篇文章里,我们将着重探讨怎样使用户体验最快:包括初始页面的下载,随后页面的下载,

 

以及随着应用渐进、内容变化而进行的资源下载。

我始终坚信这一点:对开发者来说,应该尽可能让事情变得简单。

 

所以我们青睐于那些能让系统自动处理优化难题的方法。

 

只需少许工作量,我们就能建立一举多得的环境:它使开发变得简单,有极佳的终端性能,

 

也不会改变现有的工作方式。

好大一沱

老的思路是,为优化性能,可以把多个css和js文件合并成极少数大文件。

 

跟十个5k的js文件相比,合并成一个50k的文件更好。虽然代码总字节数没变,

 

却避免了多个HTTP请求造成的开销。每个请求都会在客户端和服务器两边有个建立和消除的过程,

 

导致请求和响应header带来开销,还有服务器端更多的进程和线程资源消耗

 

(可能还有为压缩内容耗费的cpu时间)。

(除了HTTP请求,)并发问题也很重要。默认情况下,在使用持久连接(persistent connections)时,

 

ie和firefox在同一域名内只会同时下载两个资源(在HTTP 1.1规格书中第8.1.4节的建议)

 

(htmlor注:可以通过修改注册表等方法改变这一默认配置)。

 

这就意味着,在我们等待下载2个js文件的同时,

 

将无法下载图片资源。也就是说,这段时间内用户在页面上看不到图片。

(虽然合并文件能解决以上两个问题,)可是,这个方法有两个缺点。

 

第一,把所有资源一起打包,将强制用户一次下载完所有资源。

 

如果(不这么做,而是)把大块内容变成多个文件,下载开销就分散到了多个页面,

 

同时缓解了会话中的速度压力(或完全避免了某些开销,这取决于用户选择的路径)。

 

如果为了随后页面下载得更快而让初始页面下载得很慢,

我们将发现更多用户根本不会傻等着再去打开下一个页面。

第二(这个影响更大,一直以来却没怎么被考虑过),在一个文件改动很频繁的环境里,

 

如果采用单文件系统,那么每次改动文件都需要客户端把所有css和js重新下载一遍。

 

假如我们的应用有个100k的合成的js大文件,任何微小的改动都将强制客户端把这100k再消化一遍。

分解之道

(看来合并成大文件不太合适。)替代方案是个折中的办法:把css和js资源分散成多个子文件,

 

按功能划分、保持文件个数尽可能少。这个方案也是有代价的,

 

虽说开发时代码分散成逻辑块(logical chunks)能提高效率,可在下载时为提高性能还得合并文件。

 

不过,只要给build系统(把开发代码变成产品代码的工具集,是为部署准备的)加点东西,就没什么问题了。

对于有着不同开发和产品环境的应用来说,用些简单的技术可以让代码更好管理。

 

在开发环境下,为使条理清晰,代码可以分散为多个逻辑部分(logical components)。

 

可以在Smarty(一种php模板语言)里建立一个简单的函数来管理javascript的下载:

SMARTY:
{insert_js files="foo.js,bar.js,baz.js"}

PHP:
function smarty_insert_js($args){
  foreach (explode(',', $args['files']) as $file){
    echo "<script type=\"text/javascript\" SOURCE=\"/javascript/$file\"></script>\n";
  }
}

OUTPUT:
<script type="text/javascript" SOURCE="/javascript/foo.js"></script>
<script type="text/javascript" SOURCE="/javascript/bar.js"></script>
<script type="text/javascript" SOURCE="/javascript/baz.js"></script>

(htmlor注:wordpress中会把“src”替换成不知所谓的字符,因此这里只有写成“SOURCE”,

 

使用代码时请注意替换,下同)

就这么简单。然后我们就命令build过程(build process)去把确定的文件合并起来。

 

这个例子里,合并的是foo.js和bar.js,因为它们几乎总是一起下载。

 

我们能让应用配置记住这一点,并修改模板函数去使用它。(代码如下:)

SMARTY:
{insert_js files="foo.js,bar.js,baz.js"}

PHP:
# 源文件映射图。在build过程合并文件之后用这个图找到js的源文件。

$GLOBALS['config']['js_source_map'] = array(
  'foo.js'	=> 'foobar.js',
  'bar.js'	=> 'foobar.js',
  'baz.js'	=> 'baz.js',
);

function smarty_insert_js($args){
  if ($GLOBALS['config']['is_dev_site']){
    $files = explode(',', $args['files']);
  }else{
    $files = array();
    foreach (explode(',', $args['files']) as $file){
      $files[$GLOBALS['config']['js_source_map'][$file]]++;
    }
    $files = array_keys($files);
  }

  foreach ($files as $file){
    echo "<script type=\"text/javascript\" SOURCE=\"/javascript/$file\"></script>\n";
  }
}

OUTPUT:
<script type="text/javascript" SOURCE="/javascript/foobar.js"></script>
<script type="text/javascript" SOURCE="/javascript/baz.js"></script>

模板里的源代码没必要为了分别适应开发和产品阶段而改动,它帮助我们在开发时保持文件分散,

 

发布成产品时把文件合并。想更进一步的话,可以把合并过程(merge process)写在php里,

 

然后使用同一个(合并文件的)配置去执行。这样就只有一个配置文件,避免了同步问题。

 

为了做的更加完美,我们还可以分析css和js文件在页面中同时出现的几率,

 

以此决定合并哪些文件最合理(几乎总是同时出现的文件是合并的首选)。

对css来说,可以先建立一个主从关系的模型,它很有用。一个主样式表控制应用的所有样式表,

 

多个子样式表控制不同的应用区域。采用这个方法,大多数页面只需下载两个css文件,

 

而其中一个(指主样式表)在页面第一次请求时就会缓存。

对没有太多css和js资源的应用来说,这个方法在第一次请求时可能比单个大文件慢,

 

但如果保持文件数量很少的话,你会发现其实它更快,因为每个页面的数据量更小。

 

让人头疼的下载花销被分散到不同的应用区域,因此并发下载数保持在一个最小值,

 

同时也使得页面的平均下载数据量很小。

压缩

谈到资源压缩,大多数人马上会想到mod_gzip(但要当心,mod_gzip实际上是个魔鬼,至少能让人做恶梦)。

 

它的原理很简单:浏览器请求资源时,会发送一个header表明自己能接受的内容编码。就像这样:

Accept-Encoding: gzip,deflate

服务器遇到这样的header请求时,就用gzip或deflate压缩内容发往客户端,然后客户端解压缩。

 

这过程减少了数据传输量,同时消耗了客户端和服务器的cpu时间。也算差强人意。

 

但是,mod_gzip的工作方式是这样的:先在磁盘上创建一个临时文件,然后发送(给客户端),

 

最后删除这个文件。在高容量的系统中,由于磁盘io问题,很快就会达到极限。

 

要避免这种情况,可以改用mod_deflate(apache 2才支持)。

 

它采用更合理的方式:在内存里做压缩。对于apache 1的用户来说,可以建立一块ram磁盘,

 

让mod_gzip在它上面写临时文件。虽然没有纯内存方式快,但也不会比往磁盘上写文件慢。

话虽如此,其实还是有办法完全避免压缩开销的,那就是预压缩相关静态资源,

 

下载时由mod_gzip提供合适的压缩版本。如果把压缩添加在build过程,它就很透明了。

 

需要压缩的文件通常很少(用不着压缩图片,因为并不能减小更多体积),

 

只有css和js文件(和其他未压缩的静态内容)。

配置选项会告诉mod_gzip去哪里找到预压缩过的文件。

mod_gzip_can_negotiate	Yes
mod_gzip_static_suffix	.gz
AddEncoding	gzip	.gz

新一点的mod_gzip版本(从1.3.26.1a开始)添加一个额外的配置选项后,

 

就能自动预压缩文件。不过在此之前,必须确认apache有正确的权限去创建和覆盖压缩文件。

mod_gzip_update_static	Yes

可惜,事情没那么简单。某些Netscape 4的版本(尤其是4.06-4.08)认为自己能够解释压缩内容

 

(它们发送一个header这么说来着),但