Incorrect ITS in XLIFF files

[garfieldnate]

I was just working with all of the XLIFF files in the xlfsamples directories, and found some incorrect-looking ITS markup.

locqualityissue[3,4,5,9,10,11,12,13]xml.xml.xlf- these each have its:locQualityIssueProfile. It should be its:locQualityIssueProfileRef.

storagesize1html.html.xlf
storagesize[6,9]xml.xml.xlf
These each have its:storageLineBreak, which should be its:lineBreakType.

terminology1xml.xml.xlf
terminology2html.html.xlf
These each have its:termInfo, which doesn't exist in ITS. There are its:termInfoRef and its:termInfoPointer. This will require more than just changing the attribute however, because the file needs a separate node to hold the term information, and in the present .xlf file the attribute itself is holding the value.

I would send a pull request, but I think these files are generated (not sure by what).

[finnle]

I corrected the locqualityissue and storage size xliff files issues. But the term issue I think is more XLIFF/ITS recommendation question as I believe what is there currently is correct according to the current recommendation. I have emailed the group though and you are cc'd on the email

[garfieldnate]

Thanks. The current XLIFF 1.2 mapping only has its:termInfoRef and itsxlf:termInfoRef. Hopefully someone from the mailing list will have some input.

[ysavourel]

Hi Nathan,

terminology1xml.xml.xlf
terminology2html.html.xlf
These each have its:termInfo, which doesn't exist in ITS.
There are its:termInfoRef and its:termInfoPointer. This will
require more than just changing the attribute however, because
the file needs a separate node to hold the term information,
and in the present .xlf file the attribute itself is holding
the value.

I've pushed updated files today that reflect better the current mapping.
But I think terminology1html.html.xlf and terminology1html.xml.xlf were changed a while back.

Note that those files are generated from source examples that do not really have their ITS information defined (for example not all elements that should be are defined as withinText='yes', etc.), which explains some of the strange output we are getting sometimes.

for its:termInfo: I think we are missing itsxlf:termInfo in the mapping, and also a note saying that when converting its:termInfoRef to itsxlf:termInfoRef, the value of the reference may have to be changed to an IRI accessible from the output. Actually, another way would be to put the pointed content in itsxlf:termInfo, but that probably wouldn't work very well in many cases.

I would send a pull request, but I think these files are
generated (not sure by what).

It's generated with Tikal (http://www.opentag.com/okapi/wiki/index.php?title=Tikal), the latest snapshot distribution has the latest implementation of the mapping (at least what I can manage to implement).

Cheers,
-yves

[garfieldnate]

Thanks. I was getting confused because all of the other its:* attributes
are the same as in the ITS 2.0 spec, so its:termInfo would constitute a
single exception (since there is no such thing in the spec). That may
actually be a shortcoming in the ITS spec, not in the mapping, since
changing pointers into local markup seems to be the normal pattern in the
mapping.
Nathan

On Tue, Sep 24, 2013 at 2:26 PM, ysavourel notifications@github.com wrote:

Hi Nathan,

terminology1xml.xml.xlf
terminology2html.html.xlf
These each have its:termInfo, which doesn't exist in ITS.
There are its:termInfoRef and its:termInfoPointer. This will
require more than just changing the attribute however, because
the file needs a separate node to hold the term information,
and in the present .xlf file the attribute itself is holding
the value.

I've pushed updated files today that reflect better the current mapping.
But I think terminology1html.html.xlf and terminology1html.xml.xlf were
changed a while back.

Note that those files are generated from source examples that do not
really have their ITS information defined (for example not all elements
that should be are defined as withinText='yes', etc.), which explains some
of the strange output we are getting sometimes.

for its:termInfo: I think we are missing itsxlf:termInfo in the mapping,
and also a note saying that when converting its:termInfoRef to
itsxlf:termInfoRef, the value of the reference may have to be changed to an
IRI accessible from the output. Actually, another way would be to put the
pointed content in itsxlf:termInfo, but that probably wouldn't work very
well in many cases.

I would send a pull request, but I think these files are
generated (not sure by what).

It's generated with Tikal (
http://www.opentag.com/okapi/wiki/index.php?title=Tikal), the latest
snapshot distribution has the latest implementation of the mapping (at
least what I can manage to implement).

Cheers,
-yves

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-25043970
.

[fsasaki]

Thanks, Nathan. Could you close this issue and (if needed) add your shortcoming of the ITS spec to
http://www.w3.org/International/its/wiki/IssuesAndProposedFeatures#Issues_and_Proposed_Features_.28For_updating_ITS_2.0.29
so that we keep track of it?
Thanks again,
Felix

[garfieldnate]

Sure. I'll probably add everything I've found during the course of the WICS
project at once, next week some time.
Nathan

On Tue, Sep 24, 2013 at 6:30 PM, Felix Sasaki notifications@github.comwrote:

Thanks, Nathan. Could you close this issue and (if needed) add your
shortcoming of the ITS spec to

http://www.w3.org/International/its/wiki/IssuesAndProposedFeatures#Issues_and_Proposed_Features_.28For_updating_ITS_2.0.29
so that we keep track of it?
Thanks again,
Felix

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-25056310
.

[garfieldnate]

I just pulled the latest, and I still see the same problem with all of the locqualityissue files. I don't believe the mapping has been updated in any way- it doesn't mention locQualityIssueProfile[Ref], so I think the ITS-namespaced attribute should just be what is in the ITS spec, especially since it's a set pattern that anything ending in Ref is a URI.

I also found something else- in locnote[3,4,8,9]xml.xml.xlf, there are trans-units that use both <note> and its:locNoteRef (which isn't listed in the mapping, but I assume it would be fine provided the source was its:locNoteRef). Both were generated from an its:locNoteRef, and the <note> is prefixed with REF:. I understand that this is generated by a third party, but it turned out to be important for my application. The REF: suffix isn't in the mapping, and when my application tried to convert the XLIFF ITS into HTML5 ITS, I ended up with elements that had both its:locNote and its:locNoteRef, making the documents invalid.

[ysavourel]

Hi Nathan,

I just pulled the latest, and I still see the same problem
with all of the locqualityissue files. I don't believe the mapping has
been updated in any way- it doesn't mention locQualityIssueProfile[Ref],
so I think the ITS-namespaced attribute should just be what is in
the ITS spec, especially since it's a set pattern that anything
ending in Ref is a URI.

Good catch: the 'Ref' is missing: that is likely a bug. It should be there. As you say: we just use the same ITS attributes for LQI in XLIFF. I'll fix that in our writer and re-generate the examples (it may take a few days).
Note that the mapping document simply doesn't have an example with locQualityIssueProfileRef but it's useable there too since it's part of the ITS namespace. I'll try to add it in an example.

I also found something else- in locnote[3,4,8,9]xml.xml.xlf,
there are trans-units that use both and its:locNoteRef
(which isn't listed in the mapping, but I assume it would be
fine provided the source was its:locNoteRef).

We haven't really finished the discussion on how to work with locNoteRef (as you can see it's still listed as an issue in the mapping).

Both were generated from an its:locNoteRef, and the
is prefixed with REF:. I understand that this is generated by a
third party, but it turned out to be important for my application.
The REF: suffix isn't in the mapping, and when my application tried
to convert the XLIFF ITS into HTML5 ITS, I ended up with elements
that had both its:locNote and its:locNoteRef, making
the documents invalid.

The with the REF: prefix pre-date ITS 2.0 and the mapping. That was how our tools were trying to map a ITS 1.0 loc-note reference using the XLIFF .
The advantage of using is that it exists in XLIFF, so XLIFF readers can use it. But that would work only for non-inline notes.

Thoughts?
-ys

[garfieldnate]

Yes, I did notice the mapping note on locNoteRef. Whatever happens, though, I think it will be better to have separate places to put locNote and locNoteRef. Using special text content to differentiate two categories has the risk of hitting a corner case. Someday, someone, somewhere might begin their actual locNote with REF:, and they will be puzzled why it keeps being rendered as a locNoteRef. Or they'll figure it out pretty quickly, and you'll get an email asking how to begin a locNote with REF:.

[ysavourel]

The advantage of using XLIFF is that XLIFF processor will be able to use it even if they don’t support ITS.

Maybe the solution then is to use when the Locnote is not a reference, and use the ITS markup when it is a reference.

-ys

From: garfieldnate [mailto:notifications@github.com]
Sent: Tuesday, October 15, 2013 12:07 PM
To: w3c/its-2.0-testsuite
Cc: ysavourel
Subject: Re: [its-2.0-testsuite] Incorrect ITS in XLIFF files (#2)

Yes, I did notice the mapping note on locNoteRef. Whatever happens, though, I think it will be better to have separate places to put locNote and locNoteRef. Using special text content to differentiate two categories has the risk of hitting a corner case. Someday, someone, somewhere might begin their actual locNote with REF:, and they will be puzzled why it keeps being rendered as a locNoteRef. Or they'll figure it out pretty quickly, and you'll get an email asking how to begin a locNote with REF:.

—
Reply to this email directly or view it on GitHub #2 (comment) . https://github.com/notifications/beacon/nvs-vse_AjE_52DIkMMQpiz-ed5h4E6Uf1W8aJCPT6srv-oBC5atmPC3SvkpXKVB.gif

[garfieldnate]

Yes, exactly what I was thinking.