• Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).
  • Notice: Undefined index: href in oa_comment_preprocess_comment() (line 113 of /easy/www/tiffa-intranet/profiles/openatrium/modules/apps/oa_comment/oa_comment.theme.inc).

TI/A Draft is out for Discussion

Hi all

The TI/A draft is finally out and you are all invited to contribute.

The policy behind the recommendation was driven by the attempt to find a good tradeoff between sustainability and cost of conversion.

Based on the analysis in some Swiss archives (2.5 Milion files) some compressions (LZW, Fax Group 4, even new JPEG) are not allowed. If you oppose, speak up, please!

We look forward for a vivid discussion. Please submit your comments until 10th October ASAP so we can speed up the process.

Yours, Erwin Zbinden, DHLab University of Basel

AttachmentSize
draft_for_iso_standard_wip4_0.docx146.66 KB

Comments

#1

Hi all

Unfortunately the upload did not work. Document should be uploaded by tomorrow...

#2

The TI/A draft for discussion has been uploaded. You can download it from http://intranet.ti-a.org/filedepot/folder/5?fid=10

#3

I was going to comment on the prohibition on compression, but looking through the document, I find the concern is bigger than that. Other options, including YCC color and tiling, are forbidden for no obvious reason. Odd values for the value offset in an IFD are prohibited; that matches the TIFF spec but serves no purpose on modern hardware is generally ignored in practice.

The PDF/A specification prohibits features which create external dependencies or ambiguities, or which make the document mutable. TI/A seems to take a different approach, prescribing preferable features and prohibiting valid but less preferable ones. At the same time, the process and the document haven't been made public. The result, I'm afraid, is something which people will look on as an arbitrary decree and pay no attention to.

#4

Hi Gary

Which document did you read? The one of Marisa or the new one (wip4)?

Due to problems with the upload the new document has been made available yesterday at 10am. So please check

Sorry for inconvenience, Erwin

#5

The document I read was downloaded as "draft_for_iso_standard_wip4.docx.docx". The one I had earlier was a .doc file. I don't know if the double extension was an artifact of the downloading.

#6

First look, I found some issues. As a rule of thumb, the TI/A spec should be written as clearly as possible and resolve ambiguities. It should be as simple as possible and complex as necessary.

Here the issues I found so far (IMHO)

- the meaning of flag "red" in the table is not described

- the interaction between "count" in IFD and IFD-tag should be clarified, I think. A count 0 make no sense, therefore the rule should be: add a tag only, if count is at least 1

- the thumbnail paragraph is misunderstanding. I would deny thumbnails. In case of digital preservation, a thumbnail is a derivative copy. It is not a good idea to mix the level of abstractions within one filetype especially for files used as preservation master.

More in next days if I have more time to read the whole document more carefully

With best regards

Andreas

#7

Draft, Page10, IFD

In 4.1.2 "shall be at least 1 IFD". In case of TI/A we should propose "shall be only 1 IFD" -> IFD0

In answer to Gary McGath, the 16bit offset boundary makes the TIFF more robust (and errors could be detected and repaired more easily).

#8

Draft, Page 11, Count

IMHO a count of zero should not be allowed to avoid misinterpretations. There is no disadvantage, because if some tags are not yet available, you could easily point to another IFD0-offset without this tag. If you need the tag again, you could rewrite the offset to the other IFD0 *with* that tag.

#9

Draft, Page13/14

The TI/A should not contain thumbnails. Because in intro we write, that only one IFD is allowed, thumbnails are superflous.

#10

Draft, Page 15ff, Table, the meaning of column "Rating" is not described

#11

Draft, Page15, Entry "Photometric Interpretation"

Should we only list allowed values? And makes it sense to define a default value (2=RGB)?

#12

Draft, Page 15, Entry "Orientation"

Should we define a default value and make the field optional?

#13

Draft, Page 15, Entry "Resolution Unit"

For TI/A it is defined as "optional". Here I a disagree. If the fields "XResolution" and "YResolution" are mandatory, the "Resolution Unit" should als be mandatory. Otherwise define an "optional" with a default Value

#14

Draft, Page 16, Field "InterColorProfile"

This entry is here defined as mandatory, but on page 56 it is defined as optional. This is an ambiguity which should be fixed

#15

Draft, Page 17ff, Table, the meaning of column "Rating" is not described, see comment above

#16

Draft, Page 26, 5.2.1.1 NewSubFilyType

Thumbnails should not be part of TI/A, therefore only value 0 is allowed, see comment above

#17

Draft, Page 27, 5.2.1.5 Compression

TI/A should not allow Thumbnail, therefore the description should be shortened.

Also in the table we should only list all "allowed" compression methods, in case of TI/A only "no compression" should be listed

#18

Draft, Page 30, Orientation

See comment to Page 15, Entry "Orientation"

#19

Draft, Page 33, Resolution Unit

see comment to Page 15, entry "Resolution Unit", here a default value is listed as last sentence, but Tag is described as mandatory.

#20

Draft, Page 46, 5.2.3.1 DocumentName

In case of digital preservation, why should we preserve such a metadata?

It makes no sense. In my experience TIFFs with that tag often have a file path writen in this tag. We should deny that tag.

#21

Draft, Page 46, 5,2,3,2 Image Description

There is an ambiguity between Header and third sentence. In header it is listed as optional, in sentence it is defined as mandatory.

In my opinion, the tag should be optional. The sentence "This is a mandatory tag, with an allow value of '' (null character), indicating 'unknown'" should be erased.

If an Image Description is unknown, it should not exist! An "unknown" state in this field is superflous.

#22

Draft, Page 48, 5.2.3.10 DateTime

The last sentence is superflous. If the dateTime is unknown, the tag should not exists. A "000:00:00 00:00:00" is a value without any information and therefor it should not be stored in TI/A.

#23

Draft, Page 48, 5.2.3.11 Artist

Here should be added a clarification. The tag is *only* allowed for entries about the photographer. It should be stated explicitely that this tag should not be used to describe/name a scanner operator or digitization center! There should also be an hint added, that this tag contains personal data. In some countries these kind of data expects a sensitive processing by laws.

#24

Draft, Page 48, 5.2.3.12 HostComputer

IMHO not necessary to support that tag anymore. Helpful sometimes to detect errorneous software and to identify where problems come from.

But in contrast to the Tags "Software", "Make" and "Model" an error in TIFF has seldom a dependency to a specific Host or operating system

#25

Draft, Page 56, 5.2.3.32 EXIF IFD

my understanding is, that all EXIF tags should be referenced with *this* EXIF IFD as root IFD.

See also Tags "ExifVersion", "DateTimeOriginal" and so on

#26

Draft, Page 56, 5.2.3.33 IntercolorProfile

please also note the comment Draft, Page 16, Field "InterColorProfile".

There it is defined as mandatory, here it is defined as optional.

#27

Draft, Page 66/61, 5.2.3.41 ExifVersion // 5.2.3.42 DateTimeOriginal

I understand the TIFF specification that private IFDs should be used if you use a lot of private Tags do describe a specific aspect. In my opinion all EXIF data (and therefore also the DateTimeOriginal tag) should only be referenced by the EXIF IFD tag (see comment "Draft, Page 56, 5.2.3.32 EXIF IFD").

This allows us also to  define "Namespaces" to extend the TI/A in future if needed.

The Tag DateTimeOriginal should not be listed as tag in the IFD0 directly, nor the ExifVersion should.

#28

Draft, general comment

After reading the draft I am not sure, what is the reason why we need a TI/A standard. I already know, that we need a point to say "Here, we allow only this kind of files in our material" and a TI/A would help in *this* case.

But what is the goal of TI/A? Is it to cut things in  existing TIFF specifications down to be more robust?

Or should we declare the common use? What are the TIFFs which are "acceptable" to be preserved?

In the first case we could clarify the fuzzy edges in the TIFF specifications and define it in sense of "This TIFF is not only a (whatever) TIFF, but a very clean TIFF in case of TI/A standard. There is no way to misinterpret it."

#29

Hi Gary

at your #3 and #5

Thank you for your comment (and sorry for the delay in replying due to holiday break). You got the right document. You are right, there are ambiguities in the document, caused by lack of time...

There are also explanations missing for excluding tags.

Why do we exclude YCC Color? This model was developed with regard to analog tv signals and can be regarded as a sort of compression. It is complex and it violates our rule of simplicity (See introduction page 6). Furthermore analog tv is dead and the model will not be used in the future what means furture TIFF readers will not be able to read it. This explanations will be added to the recommendation.

Exclusion of tiling is the same thing: We regard tiling as a more complex implementation than strips. Also this will be explained.

Ambiguities: There are ambiguities in the draft. In our team there were several opinions and we decided to make this transparent. The idea is to let you experts comment on them and then make a decision.

A good example is CMYK color. Should we exclude it and mark it as forbidden? This color model is not complex as CIELab or YCC but what could be the disadvantages of this model? Please comment on that.

#30

Hi Andreas at #6 First look

As simple as possible and as complex as necessary: I totally agree

Meaning of flag red: Right, this has not been undefined. The column was intended for optional values, which are recommended or not recommended. Depending on the further discussion this column might be droped, at least in the mandatory and the forbidden tables. In the table related to optional tags red would mean not recommended wereas green would mean recommended.

Interaction between "count" in IFD and IFD-tag: Could you specify this? As a general rule it seems clear to me, but do you mean a particular section of the text?

Thumbnails: Basically you are totally right here. An archival master should not contain a thumbnail image, technically speaking. For new files that enter an archive this is clear.

The case of existing files in archives is more difficult. If an archive contains 1 million images and 40% have a thumbnail, should we force archivists to convert 400'000 images. What would be our arguments for this? The only argument that could make them convert would be: Your images will not be readable in n years. But is this true? I do not think so. IMHO thumbnails do not affect the readability of the main image. If I am wrong, please correct me.

#31

Andreas at #7 Draft, page10 IFD

IFD: only 1 IFD: see my comment on thumbnails.

Thanks for answering to Gary. I agree.

#32

Andreas at #11 PhotometricInterpretation

IMHO this document is not only for authors of TIFF readers and writers but also for image archivists. This means it is also backward oriented. In archives there are tons of bilevel and grayscale images, so a default value of 2 is to narrow.

#33

Andreas at #12, Draft, Page 15, Entry Orientation

You are right, this tag can be made optional. I would suggest to let the values open and no default (1, 3, 6, 8, 9 are valid). The reason behind this: The value is not critical. If we define a default value, a large group of the images will not comply to it (most values will be 1 or 9).

#34

Andreas, #13 Draft, Page 15, Entry "ResolutionUnit"

You are totally right, this is a typo and should read "mandatory" since it is in the mandatory table...

#35

Andreas #14 Draft, Page 16, Field "InterColorProfile"

Also here you're right, I forgot to delete it in the mandatory table.

#36

Andreas, at #17 Draft, Page 27, 5.2.1.5 Compression

As in "PhotometricInterpretation", I would list the various possible values and their interpretations, since this document is also for archivists and is handy for a short overview.

Allowed compression values: At first we thought of allowing only value 1=no compression. But then as we analysed existing archives comprising 3.5 million tiff files (Swiss federal archive BAR, cantonal archives of Basel and St. Gallen), we were astonished to see how many compressed files were around. With only limited time I analysed the log files of BAR which holds 1.7 million files. I could analyse 10% of the log files and found about 40'000 CCITT Fax Group 4 compressed files, furthermore several thousand LZW compressed files. (even some hundred JPEG (v=7) compressed files where around.

At iPRES in Bern (Conference about Preservation of digital files) I takled to some archivists (from Sweden, Netherlands) who confirmed this impression. While it is clear to many people that JPEG compressed files have to be converted to uncompressed, many archivists fear the work and cost to convert thounsands or hunderdthousands of Fax Group 4 files or LZW files.

This is a dilema: On one hand I would like to make the standard as clear as possible and forbid all compression. At the other hand archives have no means to decompress and convert so many files on short term.

While is should be clear that on the medium term compression should be forbidden at all on the short term I would suggest to forbid only those compression techniques that are either complex or are at risk to be unsupported in short or medium term because they are not widely used anymore.

This rationale would say:

Forbidden is: CCITT 1D, CCITT Fax Group 3, JPEG (old), JPEG (new), PackBits, any other proprietary compression technique

Allowed but not recommended is: LZW, CCITT Fax Group 4

Futher comments later today or tomorrow

Best regards, Erwin

Alternatively one could also say all compression is forbidden and give a timeline of 10 years while some compressions are still tolerated...

What is your opinion about this? My argument is very political, I know, but how do you see it from the point of stability of compressions (LZW, Fax G4)?

#37

 Erwin Zbinden writes that future TIFF readers will not be able to read YCC color. This sounds like an assumption that existing technical knowledge will disappear, and creators of future TIFF readers will have to reconstruct the format based on scraps of available knowledge. If so, the idea of archival TIFF is a bad idea in itself, and the group now working on TI/A should focus on some simpler, more viable format to survive future loss of knowledge.

But then he goes on to suggest that all compression should be forbidden. So in a sense, it's sounding as if this is a proposal for a new format, downward compatible with TIFF. The direction of discussion, which I've repeatedly opposed, is toward not excluding unsafe features, but excluding undesirable features.

I don't see any hope of success in such a project. TIFF is a well-known, widely used format. What will convince archivists that they should archive their files in TI/A rather than TIFF? Talk to them about a coming "digital dark age" and they'll respond with ridicule. They'll need new software that will exclude all undesirable features of TIFF. It's not a huge deal to do that, perhaps with a modified version of Libtiff to convert TIFF files to TI/A, but people need a reason to convert their non-TI/A files to TI/A.

I see no point and no hope of success in the venture.

#38

Dear Gary,

from my experience I do not agree with Your arguments. I would like to explain my reasoning behind it. But first let's define what "digital longterm archiving" (> 50y) means to me for images:

The bitstream representing an image that has been preserved (for a longer period of time)  has to be rendered in the future to a device in a form that it reproduces the visual aspects of an image as intended by the creator of the bitstream. 

By the way, in the context of archives of cultural heritage archives, there is nothing like a digital image – only analogue images do exist! There is of course a bitstreams representing an image, but the image is finally an analogue rendering of the digital data that can be percieved by the human eye and brain (I know that this definition does exclude pattern recognition, computer vision etc. – but for most cultural heritage objects the correct rendering for human perception is the real essence!)

Since we do not know how the IT will develop, the bitstream has to contain all necessary information to achieve this goal. In addition, the bitstream has to be documented and this documentation has to be preserved as well. Without this documentation, the bitstream is useless!

Now back to TIFF and TI/A: Indeed there are much simpler formats around, e.g.the PBM, PGM, PNM, and PPM family. However, these formats miss some important features and are not so widespread in use.

Wy use TIFF?

  1. TIFF is an ideal candidate because it's widely used and has been accepted by the archiving community.
  2. It offers a lot of options e.g. to include metadata, special formats etc. This is both (at the same time) a pro and a contra! Having so many options – many of same virtually unused – makes the format problematic. Just as an example, many years ago we digitized old B/W glass plates and calibrated our scanner to optical densities. TIFF offers a variant to store optical densities. No TIFF reader except my own software is able to render these images correctly. Still it is a fully compliant standard TIFF! For longterm archival totally useless...

So why TI/A?

  1. TI/A is not a new file format, it is completely compatible with TIFF. It is just a subset of the TIFF family. So any TIFF-compliant TIFF reader will be able to read the TI/A format.
  2. TI/A places restrictions on the many variants of possible (complient) TIFF files. Why are these restrictions necessary to ensure longevity? a) TIFF is a very flexible, versatile format which has a long history – thus it includes many options and variants which I consider problematic with regards to longevity (see above!). TI/A defines a simple and well defined subset of all possible TIFF variants which will ensure that it can be read in the (far) future – but still it is a TIFF. b) in the long history of TIFF, some options have rarely been adopted and have been replaced by opaque data-blobs which follow there own format definitions (e.g. color profiles). This can be highly problematic. ICC color profiles themselves are a complex, usually poorly understood fileformat. Using ICC profiles correctly requires high skills (if You ever tried to give an image with an uncommon ICC-profile to be printed by a professional print shop, the results are … frustrating at best (we have a lot experience with this!).
  3. TIFF is a rather complex file format. Writing a reader which covers all aspects will be very difficult, time consuming and expensive in the future – and we do not know how long the libtiff will exist! (Will future compiler still have a C-compiler ?– do You remember PL/1 and COBOL?. Without C compiler libtiff – if the code has been preserved – will be useless).

I personally  have a lot of experience  technical knowledge – especially in the field of IT – disappearing very, very quickly. Sometimes a large user community helps to preserve the knowledge (I'm currently instaling VAX/VMS on my OS X computer using the vimh emulator: there is still a big DEC community around. But how will this be in 20 years? 50 years). We know that as soon as the "standard" gets out of use, the knowledge will fade quickly, documents will be destroyed (It would not be appropriate to list here all real-world examples I experienced...). How many persons do really understand the ICC format worldwide? It is a small community

Thus I believe that TI/A is a necessity and archivists will (and do) understand it in our discussions with archivists we get a very positive feedback (and we do have a lot direct contact with small, medium and large archives!). Since TI/A is TIFF, we believe that most existing TIFF files in archives will comply already to the TI/A subset. For those who dont, a conversion in the nearer future would be a good ideal...

#39

Dear all,

Anyone knows the current status of the patents over the LZW compression?

As long as i know, Unisys U.S. LZW Patent No. 4,558,302 expired on June 20, 2003, the counter part patents in the United Kingdom, France, Germany and Italy expired on June 18, 2004, the Japanese counterpart patents expired on June 20, 2004 and the counterpart Canadian patent expired on July 7, 2004. The U.S. IBM patent expired 11 August 2006

Although, the Software Freedom Law Center says that after 1 October 2006, there will be no significant patent claims interfering with the use of LZW compression. In 2007, the company's LZW Patent and Software Information Web page stated that the "Unisys Corporation holds and has patents pending on a number of improvements on the inventions claimed in the above-expired patents.". http://web.archive.org/web/20090626052026/http://www.unisys.com/about__u...

Since June 2009, the Unisys Web site has no mention of LZW compression. 

#40
#41

Dear all

I think here must be clarified some issues about compression in the context of this document.
1) Compression ist certainly not bad, compression is a good way to reduce image volume in archives, as long as it is smartly chosen.
2) The recommendation has TWO aims:
a) EXISTING FILES IN ARCHIVES => It is important to give guidelines what features of TIFF are really problematic and must force a migration. We could only find a small number of those features but one that is really problematic is "old-jpeg" compression. Even today are many decoders not able anymore to read this format. It must be clearly said that it is not the aim of the recommendation to force any institution to migrate any TIFFs to another kind of TIFF, if not absolutely necessary. Eg it makes no sense to migrate a LZW compressed image to an uncompressed one, because no new information can be gained.
b) NEWLY CREATED FILES => If a new TIFF is created, the situation is, from my point of view, different because some features like LZW compression can only be optional. In addition the institutions must get information that eg LZW has its drawbacks as well, like complexity, so that a feature like this must be chosen smartly. JPEG compression in a TIFF makes just no sense anymore, in my eyes it could even be forbidden. The decision to select TIFF as a high quality format would be frustrated by something like JPG compression. About LZW in general: We work closely with many GLAM institutions and today nearly none is using any compression in TIFF anymore. If compression is required to reduce data volume other formats are selected, mostly a well-defined J2K, that can be perfectly combined with a IIIF server because of its features.

But to stay more general: For all features it is important to address in this recommendation both kinds of files, the existing ones in assets of GLAM institutions and newly created files.

Another thing: I personally think that the behaviour of some people on this platform is unprofessional and destructive. I don't think that some statements are fruitful or of any advantage. So please try to communicate in a proper manner to make the outcome of the initiative a positive one.

Best regards,

Peter

#42

Given the topic and the timing, I'm reasonably confident that Peter's attack on "unprofessional and destructive" comments refers to my criticisms of the approach being taken. I've said, and I continue to hold, that the dominant approach is wrong, not just particular details. People can disagree with me if they choose, but it is a serious matter to call anyone's remarks either unprofessional or destructive.

I've criticized TI/A for keeping its discussions behind closed (though not locked) doors for so long. One of the results of doing that is that people descend more freely into personal insults. Peter's comment helps to support my point.

#43

Dear Gary,

the aim of the document is: It is a recommendation for the use of TIFF in archives. This has two aspects:

Giving assistance to find TIFFs with features that are in danger.
Guides for the creation of new TIFFs

Where a) is certainly less restrictive and just addressing very problematic tags and b) gives clear recommendations for the proper use of TIFF for the application in archives. Some of the Tags are only important to mention in the context of new files and some - a subset - are problematic even today and must trigger a migration.

About constructive feedback: It means that things written in a document that has the labels „draft“ and „for discussion“ are not taken as arguments to claim a whole initiative as „I see no point and no hope of success in the venture.“, but this is just my opinion.

I would welcome very much if you could name the points more in detail you have mentioned .

Best regards,

Peter

#44

Dear Gary and all,

We don't know Peter's intentions, but first of all we have to say that what is more important for this initiative is a consensus among TIFF experts, thus discussions are more than welcome.

Second, what we understood from Peter's comment is that all the positions are respectable, but sometimes the way to express them are not professional enough. For us, expressions like "hostility", "foolish" or the Dave Rice photo are not suitable.

And finally, we indeed thing that your posts on https://madfileformatscience.garymcgath.com are very constructive, although we need concretion about your thoughts, like the list of comments posted by Andreas Romeyke.

Keep discussing please!

The TI/A Initiative.

#45

I am very grateful to see the TI/A discussions come out from behind closed doors as well. The accessibility on the discussion is really an essential aspect for such a project that mixes standardization with preservation. That said, it should take long while working in public to find some diversity of opinion or approach in regards to preservation matters. I'm discouraged so see Peter's comment (alluding to Gary) that some statements are not fruitful or of any advantage, as such debate and picking apart of the drafts is essential to gain a rough consensus that works for the community this project intends to serve.
I recommend the section "Rough consensus is achieved when all issues are addressed, but not necessarily accommodated" from RFC7282, https://tools.ietf.org/html/rfc7282#page-7.
Dave Rice

#46

Dear Dave,

Thanks for your message! We just wanted to comment that the discussions have been always open! The only thing that we thought was necessary is a free register just to better know who is behind each comment. We are happy to see that the last decision to skip register is appreciated, although now anonymous people can also post.

Please, keep discussing to address all the issues (as your referenced section says) :-)

The TI/A Initiative.

#47

I from my part agree that an open discussion is important – even in some cases when we agree to disagree :-)

Let me add a - may be personal - note: sometimes cultures are different between the continents. It looks to me as if there might be some differences between the European view and a US view towards digital archival. This is not bad at all, in contrary, it's enriching. Important is in my opinion to give arguments and to refrain from too broad generalisations.

As I wrote above: It's absolutely not bad to "agree that we disagree" in some aspects! Or – as we say here in ol' Europe: "There are many different ways leading to Rome"... :-)

#48

Dear TI/A Initiative,

I appreciate your effort. I also hope it is not too late to discuss some fine tuning of your goals.

First and foremost: Let us not define a standard for the files we have, but for the files we want to have.

I would like us to see TI/A as an unofficial TIFF successor. If TI/A gains some acceptance or even attraction outside archives and libraries: It would be most welcome. Popular file formats live longer and software support is then of no concern.

That said, TI/A 1.0 files should of cause be readable by decent TIFF supporting software.

Marcus Müller-Oertel (Landesarchiv Berlin)

#49

What a vivid discussion

I would like to summarize and especially thank Gary for attracting so much attention to our discussion. I would also like to thank Andreas for his contributions. Until now I could not answer many posts, because I have two employers and had to work for the other one last week (I only work 40% for TI/A, which explains times of silence from my side).

With much plesure I read the post of Marcus and I hope that more archivists will take part in this discussion because TI/A is about archives and archivists.

Therefore I suggest to split up the discussion now into six parts that focus on particular topics as there are:

- Information about existing files in archives

- Compression

- Color spaces (Photometric interpretation)

- Metadata tags

- Tiling vs Strips

I also would like to apologize for inconvenience caused through typos and contradictory statements. The time schedule was very narrow. But after all a draft is a draft and not a finished document.

Let me say a word about "working behind closed doors". In August I started working for this project. I analysed the latest draft and saw that it was based on TIFF rev. 6.0 and did not integrate TIFFtechNote2 and TIFF/EP. So my main purpose was to collect the relevant tags without missing anything relevant.

Internaly we had different points of view regarding compression and photometric interpretation (color models). In the end we made a desicion which we thought was reasonable but we also wanted you to comment, because we know one can see it either way. This is what happened behind the doors. Now we are discussing in the open space.

In the end we should not forget about the aim of the TI/A recommendation: It is a recommendation for archives how to get to know their TIFF files; how to identify files that are likely to be unreadable in fifty years time; and how to deal with ambiguities.

Please discuss in the above mentioned threads which I will open now and please be specific.

Thanks, Erwin