注册 登录  
 加关注
   显示下一条  |  关闭
温馨提示!由于新浪微博认证机制调整,您的新浪微博帐号绑定已过期,请重新绑定!立即重新绑定新浪微博》  |  关闭

写着玩

Bob

 
 
 

日志

 
 
 
 

Upstreaming LayoutTests  

2010-01-10 22:50:27|  分类: Chrome |  标签: |举报 |字号 订阅

  下载LOFTER 我的照片书  |
Introduction

Ideally Chromium stays as close to the base WebKit port as possible.

At the moment, there are a few significant differences that we would like to eliminate:
  • Instead of running DumpRenderTree (DRT) to run the layout tests, we use TestShell (test_shell)
  • We keep our expected baselines in our source tree instead of upstream alongside the other ports
  • We have our own python-based run_webkit_tests instead of the upstream perl-based run-webkit-tests
  • We have a number of chromium-specific tests (checked into src/data/layout_tests) that have not been upstreamed
This document outlines the approach we'll take to get all of this resolved by the end of Q1 2010. We will be complete when we can:
  • Run the WebKit regression using run_webkit_tests.py (using an upstream-only checkout)
  • Run the Chromium regression using run_webkit_tests.py (using an upstream-only checkout) - i.e., you'll be able to test the Chromium port of webkit using only code checkedout from webkit.org
We hope to convince the WebKit development team to switch from their Perl-based infrastructure to Chromium's Python-based infrastructure, for the following reasons:
  • The harness fully exploits multi-processor systems, producing near-linear speedups in running the regression
  • The codebase is more maintainable and modular
  • We can capture a wider variety of test expectations (timeouts, text-only failures, pixel failures), rather than just skipping files
  • If we use our infrastructure we can take advantage of tools like the flakiness dashboard upstream as well
Note that it is a non-goal for this project to actually switch webkit.org to our infrastructure; it is sufficient for us to be able to run the chromium.org infrastructure using code that has been upstreamed.

Plan of attack

  1. Refactor run_webkit_tests.py to isolate chromium-specific hooks (basically this involves refactoring platform_utils*.py and path_utils.py) (dpranke)
  2. Reformat run_webkit_tests.py to PEP 8 style (eseidel)
  3. Move the actual run_webkit_tests.py code to live in WebKit/WebKitTools/Scripts (eseidel)
  4. Enable WebKit/WebKitToolsScripts/run_webkit_tests.py to run chromium tests while test_shell and the chromium expectations still live downstream in src/webkit (??)
  5. Update chromium.org buildbots to use the newly-upstreamed code (??)
  6. Update webkit.org chromium buildbots to use the newly-upstreamed code (??)
  7. Modify ancilliary webkit/tools scripts (e.g., rebaseline.py) to use the newly upstreamed code. (victorw)
  8. Upstream any remaining tests in webkit/data/layout_tests/{chrome,pending} so that all of the tests live in a single tree (this simplifies 9, next). (dpranke)
    1. Modify run_webkit_tests to be able to call DumpRenderTree and handle "Skipped" lists so that we can run the tests in WebKit/LayoutTests upstream (eseidel)
  9. Add any missing functionality to run_webkit_tests.py to be able to do everything that run-webkit-tests can do. (eseidel)
  10. Implement DumpRenderTree on top of the new webkit API (yaar)
  11. Modify WebKitToolsScripts/run_webkit_tests.py to use the new DumpRenderTree implemented in (7) to run the Chromium layout tests (eseidel, yaar)
  12. Upstream the Chromium expectations into LayoutTests/platform/chromium{,-mac,-win,-linux,-win-xp,-win-vista,-win-7} (victorw)
Note that (1) - (7) can run in parallel with (8) and (11). (9) and (10) depend on (3). (12) depends on (11) and (9). (13) depends on (12).

Open Issues
  1. Design and plan for porting DRT onto webkit API (Yaar)
  2. Plan for handling multiple port variants (webkit mac and chromium mac) in run_webkit_tests.py (dpranke)
  3. Plan for handling multiple port variants of test_expectations.txt (dpranke)
Notes

We don't want to move the expectations upstream until we have completed the DRT port - it would be too painful to make changes to one if the other is in a different repository. In particular, we don't want to have to do a WebKit roll to modify test_expectations if a downstream checkin breaks something. This implies that (11), (12), and (13) pretty much need to happen at the same time.

  评论这张
 
阅读(257)| 评论(0)
推荐 转载

历史上的今天

评论

<#--最新日志,群博日志--> <#--推荐日志--> <#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇,下一篇--> <#-- 热度 --> <#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->
 
 
 
 
 
 
 
 
 
 
 
 
 
 

页脚

网易公司版权所有 ©1997-2017