burg / timelapse Goto Github PK

To reproduce: Create a recording, end capture, and immediately attempt to start a new recording.

Looks like DeterminismLog::m_isReplaying is always set to true when DeterminismLog::reset() is called, but reset is called when recording ends. The assertion is in DeterminismController::cancelPlayback(), under the PlaybackUninitialized case.

Is this assertion necessary?

[DeterminismController]        Resetting the replay log...
[DeterminismLog]               Configuring global object with determinism log: 0x0
[RiggedWeakRandom]             Configured determinism. (DeterminismLog=0x0)
[DeterminismLog]               RESET
[CacheController]              Restored the NSURLCache.
[CacheController]              Enabled the MemoryCache.
-----CAPTURE STOP-----
SHOULD NEVER BE REACHED
/Users/jake/repos/timelapse/Source/WebCore/timelapse/DeterminismController.cpp(324) : void WebCore::DeterminismController::cancelPlayback()
1   0x104200740 WebCore::DeterminismController::cancelPlayback()
2   0x1042002fb WebCore::DeterminismController::beginCapturing(WebCore::PositionMark const&)
3   0x10489d279 WebCore::InspectorTimelapseAgent::startRecording(WTF::String*)
4   0x10489d2af non-virtual thunk to WebCore::InspectorTimelapseAgent::startRecording(WTF::String*)
5   0x1047ba555 WebCore::InspectorBackendDispatcherImpl::Timelapse_startRecording(long, WebCore::InspectorObject*)
6   0x1047c62f8 WebCore::InspectorBackendDispatcherImpl::dispatch(WTF::String const&)
7   0x1047d6930 WebCore::InspectorController::dispatchMessageFromFrontend(WTF::String const&)
8   0x10484017f WebCore::InspectorBackendDispatchTask::onTimer(WebCore::Timer<WebCore::InspectorBackendDispatchTask>*)
9   0x104840583 WebCore::Timer<WebCore::InspectorBackendDispatchTask>::fired()
10  0x105624ff0 WebCore::ThreadTimers::sharedTimerFiredInternal()
11  0x105624d89 WebCore::ThreadTimers::sharedTimerFired()
12  0x1053661a3 _ZN7WebCoreL10timerFiredEP16__CFRunLoopTimerPv
13  0x7fff92618934 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__
14  0x7fff92618486 __CFRunLoopDoTimer
15  0x7fff925f8e11 __CFRunLoopRun
16  0x7fff925f8486 CFRunLoopRunSpecific
17  0x7fff88b694d3 RunCurrentEventLoopInMode
18  0x7fff88b70781 ReceiveNextEventCommon
19  0x7fff88b7060e BlockUntilNextEventMatchingListInMode
20  0x7fff8f6f7e31 _DPSNextEvent
21  0x7fff8f6f7735 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:]
22  0x7fff8f6f4071 -[NSApplication run]
23  0x1052ba7bc WebCore::RunLoop::run()
24  0x1023e5d1a WebKit::WebProcessMain(WebKit::CommandLine const&)
25  0x102301598 _ZL10WebKitMainRKN6WebKit11CommandLineE
26  0x1023014b4 WebKitMain
27  0x101f91d92 main
28  0x101f91c74 start

When replaying Facebook, actions are often requested from the determinism log in the wrong order. I copied the recording and replaying logs from a common case below, where document.cookie was requested at mark 53, when the first document.cookie call should happen at mark 92. I'm not sure why action 60 is the one released.

[DeterminismLog]         #53    CAPTURE: ResourceDidReceiveResponse(id=12; url=https://s-static.ak.fbcdn.net/rsrc.php/v2/yb/r/TPJkiQYKZml.css) 
[DeterminismLog]         #54    CAPTURE: ResourceDidReceiveData(id=12;bytes=6866) 
[DeterminismLog]         #55    CAPTURE: ResourceDidFinishLoading(id=12; finishTime=0) 
[DeterminismLog]         #56    CAPTURE: ResourceDidReceiveResponse(id=13; url=https://s-static.ak.fbcdn.net/rsrc.php/v2/yD/r/OWwnO_yMqhK.css) 
[DeterminismLog]         #57    CAPTURE: ResourceDidReceiveData(id=13;bytes=1726) 
[DeterminismLog]         #58    CAPTURE: ResourceDidFinishLoading(id=13; finishTime=0) 
[AsyncEventProxy]              Received event: type=load, target=0/node[0x7fd5be8177e0] SCRIPT
[DeterminismController]        Unrelated DOM event    0@: type=load, target=0/node[0x7fd5be8177e0] SCRIPT
[DeterminismLog]         #59    CAPTURE: ResourceHandleCreated(handleId=14; url=https://s-static.ak.facebook.com/rsrc.php/v2/yb/r/GsNJNwuI-UM.gif) 
[DeterminismLog]         #60    CAPTURE: ResourceHandleCreated(handleId=15; url=https://fbcdn-profile-a.akamaihd.net/hprofile-ak-ash4/370418_100001391660763_1360418849_q.jpg) 

...

[DeterminismLog]         #92    CAPTURE: GetDocumentCookie(ec=0document.cookie=act=1348507497847%2F1%3A1; presence=EM348507498EuserFA21B01391660763A2EstateFDutF0Et2F_5b_5dEuct2F1348506894BElm2FnullEtrFnullEtwF2640240126EatF1348507497648Esb2F0CEchFDp_5f1B01391660763F11CC; p=7; c_user=100001391660763; csm=2) 

[DeterminismLog]         #53   YIELD: ResourceDidReceiveResponse(id=12; url=https://s-static.ak.fbcdn.net/rsrc.php/v2/yb/r/TPJkiQYKZml.css)
[DeterminismController]        WAIT: 0.375 ms
ERROR: [DeterminismLog]               ERROR 0x107690ee2 != 0x107691202

/Users/jake/repos/timelapse/Source/WTF/wtf/timelapse/DeterminismLog.cpp(141) : WTF::ReplayableAction *WTF::DeterminismLog::currentAction(ReplayableAction::ReplayableType)
ERROR: [DeterminismLog]         #60    Expected replay action of type GetDocumentCookie, but got type ResourceHandleCreated (ResourceHandleCreated(handleId=15; url=https://fbcdn-profile-a.akamaihd.net/hprofile-ak-ash4/370418_100001391660763_1360418849_q.jpg))

/Users/jake/repos/timelapse/Source/WTF/wtf/timelapse/DeterminismLog.cpp(144) : WTF::ReplayableAction *WTF::DeterminismLog::currentAction(ReplayableAction::ReplayableType)

There is currently a bin/build-nightly script to create WebKit nightlies that piggyback on the system-installed version of Safari. Some things that need to happen before "nightly" Timelapse.app can be distributed:

• use a different icon than WebKit nightlies
• be tested to make sure it actually records and replays
• include license information in the .app

I don't have a repro for this.

Normally, when a page is loaded, the Inspector frontend's breakpoint manager restores the breakpoints set in local storage. This happens during replay, but not during recording, so after recording stops, it looks like all the breakpoints the user has set have been deleted. The breakpoints still exist in local storage, and are restored the first time the recording is replayed. The problem occurs even if breakpoints are not suppressed during recording.

This should mirror the decentralized approach to serialization—each action should know how to deserialize itself, given some abstract ActionDeserializer API that speaks in terms of put/get of strings, numbers, arrays, objects, bools.

Each module should implement some functionality to register the set of known actions and their "type" (currently drawn from ReplayableTypes constants). Care must be taken to properly define facade interfaces, so that no code cares about the particular serialization backend (JSON) that we use.

Tasks

• Flesh out WTF::ActionDeserializer interface
• Figure design for ActionDeserializer::registerAction(action, fptr) and module-specific callers
• Implement a lot of deserialize() methods and supporting boilerplate
• Implement WebCore::JsonActionDeserializer
• Test and hook up to an API

We need to keep a map in the frontend or backend that keeps track of the mapping between resource id's and resource URLs.

JavaScript code has access to the screen's dimensions, pixel depth, color depth, and DPI via window.screen and other methods. These cannot be changed on replay, because they are environmental characteristics. These must be memoized on capture and replay.

• window.screenX
• window.screenY
• window.screenLeft
• window.screenTop

These four APIs are implemented by WebCore::DOMWindow by asking the WebCore::Chrome object what the native frame size is. The jsDOMWindowScreenX and related methods convert from DOMWindow's int-returning methods to JavaScript values. They are generated from DOMWindow.idl.

To memoize these, it's best to add custom bindings in which the implementation of jsDOMWindowScreenX and friends consult the DeterminismLog of the JavaScript context. This is the same strategy used by JSDocument::cookie to memoize calls to document.cookie. A shared ReplayableAction with an enum field can be used to memoize the above APIs.

Sometimes when attempting to replay Facebook, replay will fail silently when attempting to reload the page.

Maybe it has something to do with the beforeunload events?

Replay log:

-----REPLAY START-----
[DeterminismLog]         #0    YIELD: Begin
[DeterminismController]        WAIT: 1.000 ms
[DeterminismController]        DISPATCH: Begin
[DeterminismLog]         #1    YIELD: DisableCache
[DeterminismController]        WAIT: 1.000 ms
[DeterminismController]        DISPATCH: DisableCache
[CacheController]              Disabled the NSURLCache.
[CacheController]              Disabled the MemoryCache.
[DeterminismLog]         #2    YIELD: InitializeFocus(focus=true; active=false)
[DeterminismController]        WAIT: 1.000 ms
[DeterminismController]        DISPATCH: InitializeFocus(focus=true; active=false)
[DeterminismLog]         #3    YIELD: InitializeWindow(size=[1015,838])
[DeterminismController]        WAIT: 1.000 ms
[DeterminismController]        DISPATCH: InitializeWindow(size=[1015,838])
[DeterminismLog]         #4    YIELD: NavigateToPage(url=https://www.facebook.com/;referrer=https://www.facebook.com/;securityOrigin=https://www.facebook.com;)
[DeterminismController]        WAIT: 1.000 ms
[DeterminismController]        DISPATCH: NavigateToPage(url=https://www.facebook.com/;referrer=https://www.facebook.com/;securityOrigin=https://www.facebook.com;)
[DeterminismLog]         #6    YIELD: ResourceDidReceiveResponse(id=1; url=https://www.facebook.com/)
[DeterminismController]        WAIT: 122.971 ms
[AsyncEventProxy]              Received event: type=beforeunload, target=0/window[0x7faf37142600] https://www.facebook.com/
[DeterminismController]        Unrelated event    0@: type=beforeunload, target=0/window[0x7faf37142600] https://www.facebook.com/
[AsyncEventProxy]              Received event: type=beforeunload, target=1/window[0x7faf375a7750] https://www.facebook.com/ai.php?aed=AQKEAqQDdx4_x-q3oa1hYh569TuJEHb3-ZxJMn1Z6kgOTKV__4va019xQutqKj80TRDBpA2TbO6i2FLGWdFHRvWUYtiEkpcFZ1HG7DeaejLFEEMvBiQLB5I6P-e0q01rFNjsiru3JiiliYPG1d0oZf8WeCdPTbAWVKkbbeSDUbQJb0R_f9Do_uTkLnU1CzBQPywxYRJ7fVBP8-r29p7XaUdNPDPvhqtAxzuGbql0rNLuFcvYKWFMa4BU1FdCpgfpzhV33Wg0qTIl1c-QGeqOlIB2DZdztFovwSPZMfG-y_RNyPvo22JQOns0FNurmzBovKIGVKmj9eMoMXcR5SbzIa-qTb2vHvAXuskGqcH9cfRs379quLCM7WQyBQlBgDvS4wMTO5gj-UhQw2uXZrlmB6MhK86ljE6IGGz0IY5y2VD4vJ9WISAcVY4BSFu4FmC8O93DBeEX7W7yCUbFPUq7ij7x5GqzTNz7xgD4M3Oy22i3WNI_jSyxPaD61k5Cb3m_46S45flsdBvZ-l0T2WNVaZpVM2mr_NXsKcBzHyRrzSVOk4L42vQGhYN9H2V0qR6V9jPHO6sNpF6LULFnQ5DSV4MVt7aQVjr6n2vYJB0xK3sKZixV1u3Mo6PxESVPGf-qJqMckNbIE9W6buxpBq7f_G5-_dmsoDKjTubdvIktumIeFOKQ3C5b_xmtfhlYUwI_lTSbMQEJtz0RkWmzHeszqZf9mwOKP9dd3bPNqL8o-OTcazDlePHmajM2ITomUdjAfBCTgx-uJDqhSb6eV8OKKQYP9rusTB-1NhXlGZ2hyfasXGwXXp7f4FzhPtrhGbbyWDmnK7LrSpHcBAv_lU_EeJP-Lx725-2T4OmtNm9gDDi7xEaak-eUWt76EMZJyzUsCXZrOoBGMYuzrtuP4CtDF1CKJ34yteY3FRQQK_vSzDfHb7jMDleDDWLQjzHDjnk9SdLtjHiSVA0E4J9GFhgCTUacxJZ5qWqZ6hHqBz9TaINzYSC6ZUBgiAAJkAUki8tEKARDPYIqMg5QqWpOcpyHYinUUUmLrwfsVB7iJj9tWsKluuvIS0V6EyShz2SgbTWL7vFw2u-Y4LQBkAtjg2zs2rWdhAgaiyxtNNUCXgfaOkgwwFSz5fEILF1avas6elwSyXgYa9khTaKZRp9aVl7eEw8OK3BES5t8_0vIigA-kiQD3_f6bCnp7h0-nBNJ8-8cKZXkTmKnXsc0NtM-0KH48tWXnLmPlJOcLOZAfijsJy5M9MWjjAFjz2M3pCo0FkHlu2XNkrVHeLy7FRzGs9d6lU4PbkpNFHYMt4DH5pl2yfAwX7l41X_rp84peJZdNBEc1LV6XigAO4UIkjd2oADXMGp1BYcGHkrjot0uZgCd6vMIdur-G3b3ic_ObJoTnIraEItg8IKZx2NBvFATh7FMVFqskXeuRyBFLONHwOGyUGCNDR_dm5-caAXcIYo2XQluA3pOOEJNdDsdFNIZs_BVMJpCz6IffA_7x_vWX0Hi84FG55MxwCxYXIlM6ptzsl-EnN4oIg1YaAOlsGW0OIAH7EyMP_2zA3ylyXH4zVa3B4vz0e_5n3XDf-jFW4joOhx1ydJm__44wYifdxjqULUqkqQRIeIdzfP8tz-69z8Q5xwg1D07zTTDi_WHRCIPfqSV3iT6uF_9lYSES8DaFY6v47DIKY6sijtD1gCKyTVltX3wgqhkbM5gxFuqv4aHQ9N5tXR4b4h2cN9zhNnEFK4g83crTIoOziMVED7B7YKCTC8TCer0zBgszHCYUs0nbebiajtSb8lT9RPscZeIUCieE9h4lP_B2baiBzlwcSzVqBx3kfjiFVjfEqgkhr3OH5HgBRZAMBcj4H3I8XwAurMFBVot
[DeterminismController]        Unrelated event    0@: type=beforeunload, target=1/window[0x7faf375a7750] https://www.facebook.com/ai.php?aed=AQKEAqQDdx4_x-q3oa1hYh569TuJEHb3-ZxJMn1Z6kgOTKV__4va019xQutqKj80TRDBpA2TbO6i2FLGWdFHRvWUYtiEkpcFZ1HG7DeaejLFEEMvBiQLB5I6P-e0q01rFNjsiru3JiiliYPG1d0oZf8WeCdPTbAWVKkbbeSDUbQJb0R_f9Do_uTkLnU1CzBQPywxYRJ7fVBP8-r29p7XaUdNPDPvhqtAxzuGbql0rNLuFcvYKWFMa4BU1FdCpgfpzhV33Wg0qTIl1c-QGeqOlIB2DZdztFovwSPZMfG-y_RNyPvo22JQOns0FNurmzBovKIGVKmj9eMoMXcR5SbzIa-qTb2vHvAXuskGqcH9cfRs379quLCM7WQyBQlBgDvS4wMTO5gj-UhQw2uXZrlmB6MhK86ljE6IGGz0IY5y2VD4vJ9WISAcVY4BSFu4FmC8O93DBeEX7W7yCUbFPUq7ij7x5GqzTNz7xgD4M3Oy22i3WNI_jSyxPaD61k5Cb3m_46S45flsdBvZ-l0T2WNVaZpVM2mr_NXsKcBzHyRrzSVOk4L42vQGhYN9H2V0qR6V9jPHO6sNpF6LULFnQ5DSV4MVt7aQVjr6n2vYJB0xK3sKZixV1u3Mo6PxESVPGf-qJqMckNbIE9W6buxpBq7f_G5-_dmsoDKjTubdvIktumIeFOKQ3C5b_xmtfhlYUwI_lTSbMQEJtz0RkWmzHeszqZf9mwOKP9dd3bPNqL8o-OTcazDlePHmajM2ITomUdjAfBCTgx-uJDqhSb6eV8OKKQYP9rusTB-1NhXlGZ2hyfasXGwXXp7f4FzhPtrhGbbyWDmnK7LrSpHcBAv_lU_EeJP-Lx725-2T4OmtNm9gDDi7xEaak-eUWt76EMZJyzUsCXZrOoBGMYuzrtuP4CtDF1CKJ34yteY3FRQQK_vSzDfHb7jMDleDDWLQjzHDjnk9SdLtjHiSVA0E4J9GFhgCTUacxJZ5qWqZ6hHqBz9TaINzYSC6ZUBgiAAJkAUki8tEKARDPYIqMg5QqWpOcpyHYinUUUmLrwfsVB7iJj9tWsKluuvIS0V6EyShz2SgbTWL7vFw2u-Y4LQBkAtjg2zs2rWdhAgaiyxtNNUCXgfaOkgwwFSz5fEILF1avas6elwSyXgYa9khTaKZRp9aVl7eEw8OK3BES5t8_0vIigA-kiQD3_f6bCnp7h0-nBNJ8-8cKZXkTmKnXsc0NtM-0KH48tWXnLmPlJOcLOZAfijsJy5M9MWjjAFjz2M3pCo0FkHlu2XNkrVHeLy7FRzGs9d6lU4PbkpNFHYMt4DH5pl2yfAwX7l41X_rp84peJZdNBEc1LV6XigAO4UIkjd2oADXMGp1BYcGHkrjot0uZgCd6vMIdur-G3b3ic_ObJoTnIraEItg8IKZx2NBvFATh7FMVFqskXeuRyBFLONHwOGyUGCNDR_dm5-caAXcIYo2XQluA3pOOEJNdDsdFNIZs_BVMJpCz6IffA_7x_vWX0Hi84FG55MxwCxYXIlM6ptzsl-EnN4oIg1YaAOlsGW0OIAH7EyMP_2zA3ylyXH4zVa3B4vz0e_5n3XDf-jFW4joOhx1ydJm__44wYifdxjqULUqkqQRIeIdzfP8tz-69z8Q5xwg1D07zTTDi_WHRCIPfqSV3iT6uF_9lYSES8DaFY6v47DIKY6sijtD1gCKyTVltX3wgqhkbM5gxFuqv4aHQ9N5tXR4b4h2cN9zhNnEFK4g83crTIoOziMVED7B7YKCTC8TCer0zBgszHCYUs0nbebiajtSb8lT9RPscZeIUCieE9h4lP_B2baiBzlwcSzVqBx3kfjiFVjfEqgkhr3OH5HgBRZAMBcj4H3I8XwAurMFBVot
[AsyncEventProxy]              Received event: type=beforeunload, target=2/window[0x7faf38750620] https://www.facebook.com/ai.php?aed=AQKrSqWVn6O6rOwgtnnmx5cjGJmdamoxeDyJEVX1_h4Wu2SEsQ2E1ouFK23URAjcZgO2Y09kfYBRl-Ic57bmljLi6YJUSUhTEvt19FlGV2HjppFXgSJSDdEZNXDXVC5To295O-ZlzWPL2teDQ5cA0wAZOSM6XJuWeBDS16Tyl8tTCPxMsDR615jGhfkHqzylrXouIOOZ5wZKmEt41r8eoE3WvmbnyafAYphOyWRzqwxcXP3U4upDpjEq0v0bzlyT-2n2kAUGpRXtvWy2vaoK1-AdYkhYEBi6L1bYS0GcWIqplxp-oj83RDQ2CtomDxnEySPQ9sg055f38Xl5Ba4wsdS32Ae0nV42LnqojNqWfaZNRNnmYoeGfMninwWq_cuwjAQ9aAE9zceftzJz9nx82qVu4D-zVvvpRE7HJDEAFqlohbSU2yrQ4L9tcm-cTDlGLj-dwB8LbAUp-p9InFj-HS_pnwVEYm0JNhrx0h36l6mm8LYHadiuXCLdgmr3DVcRKzSvu476wmncjcLqm1kYetlQy1qBWRICW_FaTBDlYfNd39cU6GJZxUmq5eUJYd6IXwC6wgVc2yRKdUzzqGujwO0rZuU9GakSO7mgOtFsJK7SCDM-tVaqFVngk7TIDZVpGLg6NnvFi6T95So9P09SEpyEOkORbebO-x4W5QVfXVa4lqDpB2SNIbe3m6qPH_dLdew5OzMVTPF0lKN1kDyi7GY2S68DModE-_btR2Vm5KWHNg
[DeterminismController]        Unrelated event    0@: type=beforeunload, target=2/window[0x7faf38750620] https://www.facebook.com/ai.php?aed=AQKrSqWVn6O6rOwgtnnmx5cjGJmdamoxeDyJEVX1_h4Wu2SEsQ2E1ouFK23URAjcZgO2Y09kfYBRl-Ic57bmljLi6YJUSUhTEvt19FlGV2HjppFXgSJSDdEZNXDXVC5To295O-ZlzWPL2teDQ5cA0wAZOSM6XJuWeBDS16Tyl8tTCPxMsDR615jGhfkHqzylrXouIOOZ5wZKmEt41r8eoE3WvmbnyafAYphOyWRzqwxcXP3U4upDpjEq0v0bzlyT-2n2kAUGpRXtvWy2vaoK1-AdYkhYEBi6L1bYS0GcWIqplxp-oj83RDQ2CtomDxnEySPQ9sg055f38Xl5Ba4wsdS32Ae0nV42LnqojNqWfaZNRNnmYoeGfMninwWq_cuwjAQ9aAE9zceftzJz9nx82qVu4D-zVvvpRE7HJDEAFqlohbSU2yrQ4L9tcm-cTDlGLj-dwB8LbAUp-p9InFj-HS_pnwVEYm0JNhrx0h36l6mm8LYHadiuXCLdgmr3DVcRKzSvu476wmncjcLqm1kYetlQy1qBWRICW_FaTBDlYfNd39cU6GJZxUmq5eUJYd6IXwC6wgVc2yRKdUzzqGujwO0rZuU9GakSO7mgOtFsJK7SCDM-tVaqFVngk7TIDZVpGLg6NnvFi6T95So9P09SEpyEOkORbebO-x4W5QVfXVa4lqDpB2SNIbe3m6qPH_dLdew5OzMVTPF0lKN1kDyi7GY2S68DModE-_btR2Vm5KWHNg
[DeterminismController]        DISPATCH: ResourceDidReceiveResponse(id=1; url=https://www.facebook.com/)

When starting replay, the document's size should be initialized to match the starting size during capture.

When replaying memoized actions, no distinction is made between calls from injected scripts (from Inspector) and normal scripts.

If we can determine from the ExecState whether the script was injected, we could obtain values the normal (non-deterministic) way when calls are received from the inspector console.

Alternatively, maybe we could return memoized values without advancing the replay position. This would be more complicated, but would allow the user to explore a recorded execution state using the inspector console.

Due to upstream changes, it's no longer possible to selectively cherrypick the "Lock" icon and status message element and inject it into the gutter. Now, when the gutter is created, no panel objects have yet been constructed, so there are no elements to stick into the gutter.

To fix this, we could try one of the following ideas:

• move the statusMessage element to be independent of the Timelapse panel, and move lock to the left side
• find some other reliable time to append both of these elements to the inspector's gutter

For some reason, the text entered per keypress is incorrect. It's also logged in debug mode wrong. It looks like an encoding issue.

I'm not sure about this, but I would guess that our replayed user input actions will not correctly click through things like window.alert. If this is the case, then we should memoize the results of these dialogs similarly to context menu or document.cookie. Memoization will make the dialog box not pop up on replay, but we don't have much choice if the dialogs are native (OS) elements.

Simple repro steps. This was in a debug build.

Sliders in the heatmap and overview no longer work, due to upstream Web Inspector API changes.

The merge commit 20c5e83 pulled in these changes.

To fix this, we'll need to replace uses of WebInspector.elementDragStart and WebInspector.elementDragEnd with WebInspector.installDragHandle. See uses of installDragHandle in SplitView.js.

This will require some additional refactoring of the overview's "zoom selection" code, which was previously implemented with these helpers. Instead, the callback for "dragging is starting" needs to decide whether dragging should actually start, or not (say, if the user actually clicked on another element that may handle mouse events.)

In the current setup, it's difficult to see where exactly user input is directed, especially for keyboard events and mouse events that don't have visible style effects.

Upstream WebKit has just landed general overlays, so it will be possible (one day) to insert arbitrary DOM content to highlight elements on the page. For inspiration, see some of the cursor-highlighting features in Camtasia, or ask Andy about prior research.

If paused at a breakpoint or input, navigating away from the recorded page can cause unexpected behavior. If actions like these are performed, the correct thing for the replay engine to do is abort the replay, unlock input, and chill out.

We need to create some new factory methods that create the records. Also, since we can stop on arbitrary network actions, we need to increment the markindex for each of these actions (somehow).

Sometimes, on pages with iframes, replaying a FocusSetActive event will not dispatch a focus or blur DOM event. Replay will then hang on the next event, waiting for the focus/blur event to be dispatched:

[DeterminismLog]         #86   YIELD: FocusSetActive(to=active)
[DeterminismController]        WAIT: 7.612 ms
[DeterminismController]        DISPATCH: FocusSetActive(to=active)
[DeterminismLog]         #87   YIELD: ResourceDidReceiveData(id=52;bytes=64075)
[DeterminismController]        Waiting to dispatch next action (current: 15@; target: 16@).

For future awesomeness, I'd like to be able to put the top overview into a drawer. The first step towards that is to split the table from overview and timelines, and associate the status bar items with the overview.

The timeline has an example of two views split by a horizontal bar.

The underlying AppleScript open-xcode-and-attach-to-process.scpt needs to be updated to do some slightly different actions for 4.3.

When <WebKit2/WebPage.cpp> handles context menu events, it performs a hit test to determine whether it should dispatch the event to the main frame or another frame. Currently, we only dispatch the event to the main frame, which will not be the correct behavior on sites with multiple frames.

This is the final step after #56, #57.

Right now there are two types of drawer-using views: console, and others.

• Console has its own global status bar button to show/hide, and is hidden when Console panel is opened.
• Advanced search (Cmd-alt-F) and local source modifications (right-click the editor) is an example of the other kind, which pops over the current drawer and restores it when dismissed with the [X] button.

We probably want to use the console-like drawer view, since Timelapse isn't context-dependent—it can be used in several different tabs, and there's no obvious local context from which it should be launched.

We'll have to fix several hardcoded behaviors for the console view related to restoring after the latter view type is dismissed, among other things.

If the recording does not include any events which should be pushed to the frontend, the frontend ends up with a replay record of length 0, but assumes a positive length. To the user, Timelapse appears to hang, displaying the message "Recording... Click again to stop."

This is currently easy to replicate--just record Tetris and stop recording without interacting with the page. Once network records are pushed to the frontend (#32), this will almost never happen (maybe if the user tries to record about:blank?). Still, it would make more sense to display some message about nothing being recorded and return to the initial blank screen with the "Click to Record." message.

Screen dimensions and other information can be accessed by the window.screen API. This returns a Screen object which has various properties.

Screen is implemented by WebCore in page/Screen.{h,cpp} and has an auto-generated binding called JSScreen.{h,cpp}.

There are two general ways to implement this:

• create custom bindings for all methods of the Screen object (the approach of Issue #21)
• create new Screen subclasses that are created during capture or replay (either inside DOMWindow::screen or by routing that method through a Timelapse proxy). These will either capture all return values to the action log, or pull return values from the log.

I've seen this happen both on Facebook and the online version of Space Invaders, which has an iframe for ads. Resource callbacks go through CapturingResourceHandleClient after recording has stopped, causing an assertion failure.

[DeterminismLog]          Writing  1069: GetCurrentTime
[DeterminismLog]          Writing  1070: EnableCache
[DeterminismLog]          Writing  1071: EndSentinel
[DeterminismController]        Resetting the replay log...
[DeterminismLog]               Configuring global object with determinism log: 0x0
[RiggedWeakRandom]             Configured determinism. (DeterminismLog=0x0)
[DeterminismLog]               Configuring global object with determinism log: 0x0
[RiggedWeakRandom]             Configured determinism. (DeterminismLog=0x0)
[DeterminismLog]               Configuring global object with determinism log: 0x0
[RiggedWeakRandom]             Configured determinism. (DeterminismLog=0x0)
[DeterminismLog]               Configuring global object with determinism log: 0x0
[RiggedWeakRandom]             Configured determinism. (DeterminismLog=0x0)
[DeterminismLog]               RESET
[CacheController]              Restored the NSURLCache.
[CacheController]              Enabled the MemoryCache.
-----CAPTURE STOP-----
ASSERTION FAILED: capturing()
/Users/jake/repos/timelapse/Source/WebCore/timelapse/DeterminismController.cpp(445) : void WebCore::DeterminismController::capturePageInput(WebCore::DispatchableAction *)
1   0x1029708fd WebCore::DeterminismController::capturePageInput(WebCore::DispatchableAction*)
2   0x103ef86b1 WebCore::CapturingResourceHandleClient::didReceiveResponse(WebCore::ResourceHandle*, WebCore::ResourceResponse const&)
3   0x1039fdfff -[WebCoreResourceHandleAsDelegate connection:didReceiveResponse:]
4   0x7fff89883e83 ___NSURLConnectionDidReceiveResponse_block_invoke_1
5   0x7fff89883e00 _NSURLConnectionDidReceiveResponse
6   0x7fff9176dde8 URLConnectionClient::_clientSendDidReceiveResponse(_CFURLResponse*, URLConnectionClient::ClientConnectionEventQueue*)
7   0x7fff9181d9e6 URLConnectionClient::ClientConnectionEventQueue::processAllEventsAndConsumePayload(XConnectionEventInfo<XClientEvent, XClientEventParams>*, long)
8   0x7fff9181db0a URLConnectionClient::ClientConnectionEventQueue::processAllEventsAndConsumePayload(XConnectionEventInfo<XClientEvent, XClientEventParams>*, long)
9   0x7fff91748389 URLConnectionClient::processEvents()
10  0x7fff9174822e MultiplexerSource::perform()
11  0x7fff9090f4f1 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__
12  0x7fff9090ed5d __CFRunLoopDoSources0
13  0x7fff90935b49 __CFRunLoopRun
14  0x7fff90935486 CFRunLoopRunSpecific
15  0x7fff86ea64d3 RunCurrentEventLoopInMode
16  0x7fff86ead781 ReceiveNextEventCommon
17  0x7fff86ead60e BlockUntilNextEventMatchingListInMode
18  0x7fff8da34e31 _DPSNextEvent
19  0x7fff8da34735 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:]
20  0x7fff8da31071 -[NSApplication run]
21  0x103a294ac WebCore::RunLoop::run()
22  0x100b53d1a WebKit::WebProcessMain(WebKit::CommandLine const&)
23  0x100a6f598 _ZL10WebKitMainRKN6WebKit11CommandLineE
24  0x100a6f4b4 WebKitMain
25  0x100833d92 main
26  0x100833c74 start

The Web Storage APIs (Mozilla MDN description, W3C draft) allow websites to store persistent key/value pairs.

These API calls need to be made deterministic. The two obvious technical approaches:

1. Save the initial state of localStorage/sessionStorage for each page as it is loaded. On replay, re-initialize localStorage state to the saved values.
2. Memoize/intercept all calls and return values for the APIs. On replay, block API calls from reaching the underlying backing store, and instead use values from the recording.

It is unclear which approach is more desirable, so it would be good to attempt both and compare their space overhead for different workloads. A similar tradeoff exists for other persistence APIs, such as WebSQL and Application cache.

Safari 6 has major changes, including a different web inspector UI. We should make sure that Timelapse still works with the classic UI, so that we are able to ship with the latest version of Safari and OSX.

I have a Time Machine backup, so I'll do this over the weekend or some other time when I can spend a few hours restoring if things don't go well.

During capturing, platform keyboard events are captured inside WebCore::EventHandler::handleKeyEvent. Normally, a step in this function is to ask the NSResponder (via WebKit2::WebView) to interpret the key currently being processed by using an IME. But, during replay, the "key currently being processed" has no native (NSEvent) equivalent, so there is no way to recreate the IME interaction directly. Besides, IME may be nondeterministic.

A simple workaround is to simply not give the IM a chance to handle the key event, and always input directly.

Possible Solution: might be able to capture all Editor::insertText commands that happen as a result of the call into the IM, and then at replay time simply run these insertText actions again. Of course, this would imply a new memoized action type, and some sort of additional flag to PlatformKeyEvent.

Some code paths that are routed through AsyncEventProxy don't seem to be necessary any more, such as readystatechange. Investigate whether replay breaks when these are removed. In particular, figure out whether load is actually asynchronous when network replay is deterministic.

Some network actions have the URL in the preview column, and appear to be linkified, but actually clicking the link does nothing. This seems to only be the case for resources that have a leading forward slash. Perhaps the linkifier is being called incorrectly in these cases.

Test case: http://matthaynes.net/playground/javascript/glow/spaceinvaders/

Screenshot: http://imgur.com/Otxh2 (links without an underline are the broken ones)

Most of the frontend's scalability problems arise from the extremely inefficient table (DataGrid.js). It needs to use an approach like SlickGrid or CodeMirror to scale to 1000's of inputs.

Tasks

• Look at SlickGrid and similar projects to see whether they can be included in inspector without much refactoring
• Adapt or reimplement

It needs to not talk about network proxying, since we don't use a proxy any more.

Usually, no ResourceWillSendRequest records are created during a recording. Some sites cause them to be created for resources loaded after the initial page load (e.g. mozilla.org), but most resource loads apparently don't send the willSendRequest callback through the ResourceHandleClient.

This doesn't seem to cause problems with replay, but it might be desirable to have the RequestedResource records available in the frontend for debugging.

Timelapse recordings should be exported and imported on-demand only. So, there needs to be API calls in the Inspector protocol which solicit a filename to import/export.

An example of how to implement this, for Inspector frontend and backend parts, is in the Timeline panel's ability to import/export timeline data.

The actual JsonActionSerializer also needs to be modified to take a filename argument, and fail more gracefully.

InterpretedKeyCommands needs to be located in WebCore, even though it is only used from WebKit2 code.

It will be a lot easier to deserialize all actions if they are defined in WebCore, JavaScriptCore, or WTF. If not, there has to be some initialization code in WebKit/WebKit2---and these modules are otherwise not involved in Timelapse, aside from calling APIs with modified signatures.

When replay is unlocked, DOM timers should re-animate at the normal callback speed.

Currently, DOM timers created by the page during replay remain in special Timelapse mode, and do not fire based on callbacks from the platform event loop.

Right now, we only render circles for the breakpoints that are active. It would be better to render circles for all breakpoints in a faint color or outline, and then on top render circles that just represent the active breakpoints.

We need some sort of API to push/pop user-chosen time ranges of the overview, so that they can be restored after automated actions that change the active range. For example, if clicking and holding on a circle would zoom to show the circle's range only, the user needs a way to back out the zoom if it was unintended.

This API needs to be separate from setters of the zoom range, since the setter could be called many times by animations of the zoom range (in the future).

Upstream changes have apparently broken breakpoint radar, and seemingly, the ability to hit breakpoints at all. This was introduced with the 20c5e83 merge.

More investigation is required.

Upstream Webkit2 has added an out-of-process Network loader that schedules network requests. This uses IPC just like UIProcess/WebProcess. It's not clear whether we can shim this easily. For now, we should make sure it is disabled.

Per discussion at the end of Issue #41, we may be missing some checks as to whether the determinism log is currently "active". If not "active", then the script execution context was started by user actions we want to ignore, such as Inspector console commands.

The easiest way to check for this is to find all uses of DeterminismLog; main users will be memoized actions.

We need a meta "controller" view that displays the main widgets for controlling record/replay. This is the same view that will go into the drawer.

Creating a meta view abstracts away the details of which (record or replay) control is active. It also lets us associate the gutter status bar buttons with the active controls instead of the entire panel (which may not be visible).

This depends on #56.

When a user resizes the browser window, this fires the DOM resize event (part of DOM Level 3 Events). Resizing the page may alter layout dimensions (and thus, JavaScript calls to animate widgets) and affect determinism.

This needs to be recorded and replayed by routing non-programmatic (i.e., manual) window resizing calls through AsyncEventProxy. (Despite the fact that the resize event is synchronous, it is implemented asynchronously by WebKit—it fires through the DocumentEventQueue after any resize-induced layout reflow has been completed.)

It doesn't seem likely that we could force the window to be the same size during replay, but we should be able to change its size using DOMWindow.resizeTo. This should be sufficient, though a note cautioning against resizing during replay should be added to the user FAQ.