Provide an example of complete iteration in the README about proposal-intl-segmenter

[littledan]

Sure, PRs welcome!

[gibson042]

I can't PR this one myself because "TODO: define possible values, not part of UTS" doesn't provide sufficient information for me to understand the behavior. That's part of why I want a more complete demonstration. 😉

[littledan]

Now, the values of breakType are written in the specification, so that TODO is out of date.

[gibson042]

I saw that, but I am still not clear on how everything is intended to manifest or what exactly breakType conveys for all granularities. Is this correct?

let segmenter = new Intl.Segmenter("fr", {granularity: "word"});
console.log(...segmenter.segment("Ceci n'est pas une pipe"));
// logs the following to console:
// { segment: "Ceci", index: 0, breakType: "word" }
// { segment: " ", index: 4, breakType: "none" }
// { segment: "n'est", index: 5, breakType: "word" }
// { segment: " ", index: 10, breakType: "none" }
// { segment: "pas", index: 11, breakType: "word" }
// { segment: " ", index: 14, breakType: "none" }
// { segment: "une", index: 15, breakType: "word" }
// { segment: " ", index: 18, breakType: "none" }
// { segment: "pipe", index: 19, breakType: "word" }

[littledan]

I thought it would be the other way around, of indicating the type of the break it found and what caused that, but I am not positive. Btw you can test this in Chrome Canary or Node nightly versions if you pass a flag to enable the feature.

[gibson042]

I really don't care what any given early implementation does, I care about the what the proposal intends. The current README is both incomplete and also in disagreement with the spec text (which doesn't allow for breakType: "letter"), and I seem to have guessed wrong in my attempt to divine that intent. Could you please update whatever is necessary to resolve this confusion?

[aphillips]

Usually the break type of every segment will be the break type of the iterator itself. That is, iterators of breakType: word produce word breaks. In the example @gibson042 gives above, the spaces have word breaks (see UAX#29 here).

[littledan]

OK, if word breaks will always give "word", we should probably just use undefined as the breakType. The place where the breakType is really needed is soft vs hard line breaks; I just included it in the others out of consistency and because ICU does it.

[littledan]

The reason I suggested testing it in Chrome is because @FrankYFTang implemented this version with care and based on his subject matter expertise, so I believe he chose good details for these sorts of semantic edge cases.

[FrankYFTang]

hi @aphillips long time no see (10 years ?) :)