Process2014 introduced an AC ballot for CR transitions and now we have a ballot open for a year about w3process

[dwsinger]

I like the direction of Leonie's thinking, and I agree with her and Dom's analysis.

But I wonder if the goals could not be achieved by simply asking people to respond to the CR transition announcement. "If you feel that this specification should not be proceeding towards Recommendation, we encourage you to respond to this announcement with your reasons, as early notification helps us all." (I don't think we need to ask the other question, whether you support it, because proceeding along the track is the assumed action).

[dontcallmedom]

as a staff contact, I too find confusing these open ended questionnaires, and can say that at least they're not well incorporated into our practices today.

I think it would be useful to distinguish:

• what we want to accomplish
• how the process codifies it
• how well the implementation of the process works

@chaals points to 2 or 3 objectives:

1. allow for AC to ask to stop work early
2. allow for AC to push for faster completion
3. allow to remove PR as a stage

While it's clear there needs to be a formal review for (3), it's also clear that the current process doesn't enable that objective since PR is still required.

It's less clear to me that a formal review is needed for (1) and (2) - they could be accomplished with a less formal approach (e.g. as email responses to the call for implementations).

In any case, I think it's also clear that the current implementation of the formal review approach is broken - here are the problems I perceive:

• the WBS questionnaire is open-ended, cluttering the "my questionnaire" list that some AC Rep use as their todo list
• I don't think we have internal workflows to ensure the questionnaires end dates get updated as the document moves forward or backward (I could be wrong on that)
• I don't think we have reliable workflows to handle negative reviews in these questionnaires early - at least I don't since I'm more used to be tracking short-term questionnaires

I think these issues could be reasonably easily addressed. But it's probably worth first understanding if we want to do it the formal way or not, and then make sure we have a more complete list of issues that the current implementation raises.

[dontcallmedom]

I read @dwsinger suggestion (which I support) to be without surveys - just asking people to respond to the announcement email.

[dwsinger]

OK, I am on the chairs list, and there I see the transition requests. Maybe we can simplify this and simply have the CR transition request sent to the AC forum as well, and indicate in it that if any AC rep has doubts or objections, they should raise them with the team 'immediately'? That gives the AC the heads-up of a CR request, and then we'll have the formal ballot to confirm "yep, they've made another spec we'll have to obsolete one day" at PR?

[vfournier17]

When I clicked through all of the links, here's what I found:
https://www.w3.org/2002/09/wbs/33280/ric-1-2017/

It looks like it was closed on 4/18/17. So perhaps we can close this one?

[dwsinger]

On Jul 12, 2017, at 10:07 , Virginia Fournier ***@***.***> wrote: When I clicked through all of the links, here's what I found: https://www.w3.org/2002/09/wbs/33280/ric-1-2017/ It looks like it was closed on 4/18/17. So perhaps we can close this one?

The problem in the process remains. I have a ballot now that’s open until the fall. ugh Chaals thinks this is a feature. I think it’s a bug. I am inclined to start voting “no, the document as presented is not ready”, which is an honest reply to the question asked. David Singer Manager, Software Standards, Apple Inc.

[michaelchampion]

I agree this is a bug. The point of the change IIRC was to notify AC members that there is a CR document that needs wide review. These CR review ballots are a significant source of confusion.

Are we talking about this language in 6.1.2 stating that at CR, "begin formal review by the Advisory Committee, who may recommend that the document be published as a W3C Recommendation, returned to the Working Group for further work, or abandoned." I don't think that implies that a WBS ballot be opened, unless "formal" == "WBS ballot". Or is there some other language introduced in 2014 that triggered these ballots?

Would the benefit of telling AC Reps that there is a wide review opportunity be accomplished with less confusion if the Process said "provide an opportunity for review by the AC, who may ..."? Maybe the Process should suggest some other means of conveying an opinion that the work was not really ready to go to Rec besides a ballot that nobody seems to vote on and doesn't have an actual effect on spec progression.

[chaals]

@michaelchampion:

The point of the change IIRC was to notify AC members that there is a CR document that needs wide review.

No, the point is that it allows AC reps to say, early "yeah, get on and ship this already" or "we don't think you should keep polishing, we want to stop wsting time on this dead-end project now".

(Wide technical review is required to get into Candidate Recommendation)

It came about with a proposal to drop the formal Proposed Rec phase, and when that was shelved it was still there.

The implementation has been terrible - for a couple of years it wasn't done at all, and then it led to a set of ballots open for random lengths of time, whereas the requirement is a ballot that opens at CR and closes at the end of PR - and says so.

[dwsinger]

I feel fairly strongly that when we ask a formal question in a ballot, the material needed to answer the question should all be available. The ballot is open for a period in order to allow the voters to study the question and the materials and make an informed answer. These ballots don't seem to meet that measure. In addition, if, as Chaals says, it's to allow people to say "damn, ship it now", I fail to see how that works. The ballot is advertised as being open until November 2018; we can't terminate it early (maybe all the supporters voted early and the people studying the real problems in it are still studying).

So, I don't understand what this is supposed to do, I think; I don't understand what it is doing; and I don't like ballots hanging over me for months when I can't, in fact, answer them. I am tempted to start voting "No, this is not ready for advancement."

So I, at least, need some education. Maybe we should discuss at TPAC and explain to our voting body what's happening and how it's supposed to work?

[dwsinger]

Here's an example of a ballot that is confusing:

W3C is pleased to announce the re-publication of HTML Media Capture as Candidate Recommendation.

As per current W3C Process, this begins the Review Period during which Advisory Committee members may indicate whether they endorse this as a W3C Recommendation or object to its eventual advancement.

Even if it is widely endorsed and has no objections, it must still go through the Proposed Recommendation stage for at least 4 weeks, and it might have substantive changes which require additional versions published as Candidate Recommendations. Because of this, many AC Representatives choose to wait until Proposed Recommendation to make their decision. There will be another AC announcement when that publication occurs.

If you choose to vote now, you may return and change your vote at any time before the end of the review period. The review period will be extended to end at least four weeks after the Proposed Recommendation is actually published.

Since this new process requires this poll to apply across multiple distinct versions of the spec, we are omitted information that would only apply to one version. Please navigate to the appropriate version to see the abstract, change log, link to issues list, and other information.

from https://www.w3.org/2002/09/wbs/33280/html-media-capture-cr/

[chaals]

I don't think the example is confusing. I think it states what the ballot is for, how it works, and, notes that there will be a reminder if you're not ready to answer now.

That said, I think the implementation of this has been confusing, not least for those who had to do it, partly because it took a long time.

I have used this option as an AC rep - so far to request that new versions of specs that are maintained be moved forward rather than trying to produce one true perfect version, rather than foreshadowing an objection to some Recommendation.

[chaals]

Removing PR is not a goal - it was something we were considering 5 years ago and decided against. Wisely, IMHO.

The rest of what you say - including the fact that there still seems to be weak implementation - makes a lot of sense to me. If we want to keep this, which is my own preference, then I think we need to address the implementation issues you outline, and explain more clearly to AC reps what the Process says and what it is meant for and how it will work...

[LJWatson]

@chaals

"I don't think the example is confusing. I think it states what the ballot is for, how it works, and, notes that there will be a reminder if you're not
ready to answer now.">

I think you might be viewing this from the point of view of someone who is intimately familiar with the Process. For AC reps who are less familiar, and especially for those whose time is limited (recalling @edent's comment at the AC meeting in Beijing), I think it's likely to be different.

@dontcallmedom

"as a staff contact, I too find confusing these open ended questionnaires, and can say that at least they're not well incorporated into our practices today.">

+1 as both chair and AC rep.

With the approach as described in the HTML Media Capture survey, it's conceivable that a ballot could remain open indefinitely. Of the 61 specs in CR, 20 have been there for more than 18 months (3 have been there for around 1.5 decades). That's a lot of checking on specs and/or surveys in case your opinion as AC rep changes, not to mention a lot of clutter on the open surveys page.

Of @Chaals' three objectives, it isn't clear how 2 could be accomplished without removing or shortening PR. As @dontcallmedom notes, removing PR isn't an option, so unless reducing it to fewer than 4 weekks is a possibility, it seems that neither 2 nor 3 is practical.

It then seems the AC need to answer two questions:
1. Should progress on this spec be halted?
2. Should this spec become a Recommendation?

If a survey to ask these questions opens when the spec enters CR, is there any need for it to remain open through PR? Given that nothing substantive can change during CR, any reasons a member might have for either stopping or supporting progress should exist before the spec transitions to PR.

If there isn't any reason to ask during PR, or any reason why we'd need to ask a second time during PR, then a simple option might be to open the survey as the spec enters CR and leave it open for a standard four weeks (to avoid extending it based on an unknown time in CR).

[LJWatson]

So the survey to ask AC reps whether they support the specification progressing to Recommendation would open as the spec enters CR, and remain open for a standard four weeks (being the minimum time permitted in CR)? If so, WFM.

[LJWatson]

Given that there has to be a formal AC review at some point, it makes sense to open that survey when the spec reaches CR instead of waiting until it reaches PR.

[dwsinger]

I don't think it does the way we're doing it. I think we have three choices
a) the very long AC ballot we're doing today
b) two standard-length ballots, entering CR and then at PR
c) notification to the AC at CR (email) asking if anyone objects, sees problems, etc., and the formal ballot at PR

I don't like (a), and I suspect (b) is excessively heavy.

I apologize for the senile moment, but before we move on this...what was the modus operandi before this change in 2014?

[LJWatson]

Agree with your assessment of A and B. Curious what the benefit/reason for waiting until PR for the formal survey would be though?

If memory serves, the reason for introducing the extended survey was to enable the AC to feedback/object as early as possible in the transition process.

[chaals]

@LJWatson your memory serves:

the reason for introducing the extended survey was to enable the AC to feedback/object as early as possible in the transition process.

@dsinger before Process 2014 there was one AC review, which began at PR. In the 90s the Working Group just went ahead based on its charter. In the early 2000s Candidate Recommendation was introduced and transition reviews became more codified - but still only done by the WG and director.

I agree that your (b) is painful, and don't support trying to introduce it.

[michaelchampion]

Agree with @dwsinger -- let's keep this part of the team procedure, not the process document: The Team should inform the AC about CR transitions, point to the appropriate GitHub repo / public mailing list, and invite comments about accelerating or discouraging transition to PR to go to that forum. Also, they could invite members to join the WG to influence its decision.

[dwsinger]

Closing as addressed by #105

[plehegar]

It looks like the Process wasn't cleaned up enough. The Process still says:
[[
Candidate Recommendation (CR)
[...]
begin formal review by the Advisory Committee, who may recommend that the document be published as a W3C Recommendation, returned to the Working Group for further work, or abandoned.
]]
https://www.w3.org/2018/Process-20180201/#RecsCR

See also
https://lists.w3.org/Archives/Public/w3c-wai-gl/2018JanMar/1426.html

[dwsinger]

sigh. can you propose a pull request?

[chaals]

unassigning myself. As far as I can tell @LJWatson's PR addresses this, and it's a question of agreeing and applying it.