Skip to content

Commit 2a8cf2e

Browse files
committed
Merge pull request #5 from rolkar/feature/doc
Feature/doc
2 parents d21b2a9 + 6470c6b commit 2a8cf2e

File tree

7 files changed

+107
-174
lines changed

7 files changed

+107
-174
lines changed

README

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,21 @@
11
This project contains tools for manipulating X3F files from Sigma cameras.
22
See doc/readme.txt and doc/copyright.txt.
33

4-
To run the code:
5-
Download the code
4+
-------
5+
COMPILE
6+
-------
7+
8+
Download the code.
69
run 'make'
710
<wait>
8-
The bin directory contains binaries that can be run directly from the command line. Alternatively, you can create binaries for multiple platforms by calling:
11+
The bin directory contains binaries that can be run directly from the
12+
command line.
13+
14+
-------------
15+
DISTRIBUTIONS
16+
-------------
17+
18+
Alternatively, you can create binaries for multiple platforms by calling:
919

1020
make dist-all
1121

@@ -15,16 +25,40 @@ as well as the platform specific calls:
1525
make dist-32
1626
make dist-64
1727

18-
If you don't want to install all the dependencies, you can also install and run in a vagrant. Doing so requires downloading Vagrant from http://www.vagrantup.com. Once you've done so, the commands to run the code are:
28+
The distribution will be found in the file
29+
30+
dist/x3f_tools-<VER>-<PLATFORM>
31+
32+
The <VER> part is either the git hash, or if there is a tag on current
33+
commit, that tag.
34+
35+
-------
36+
VAGRANT
37+
-------
38+
39+
If you don't want to install all the dependencies, you can also
40+
install and run in a vagrant. Doing so requires downloading Vagrant
41+
from http://www.vagrantup.com. Once you've done so, the commands to
42+
run the code are:
1943

2044
vagrant up
2145
<wait>
2246
make
2347

48+
-----
49+
TESTS
50+
-----
51+
2452
If you want to run tests, type:
2553

2654
make check
2755

28-
The tests require that a python virtual environment be installable on the current system. That precursor can be met either by installing `pip` and `virtualenv` on your local box, or by running through the vagrant. Once that requirement has been met, the makefile will ensure that the other packages are installed properly.
56+
The tests require that a python virtual environment be installable on
57+
the current system. That precursor can be met either by installing
58+
`pip` and `virtualenv` on your local box, or by running through the
59+
vagrant. Once that requirement has been met, the makefile will ensure
60+
that the other packages are installed properly.
2961

30-
The other requirement for the tests is to pull down sample images from the x3f_test_files repository. This is a one-time download of about 90 mb of Sigma images that are used to run tests.
62+
The other requirement for the tests is to pull down sample images from
63+
the x3f_test_files repository. This is a one-time download of about
64+
90 mb of Sigma images that are used to run tests.

doc/changes.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
x3f_053: Erik and Roland Karlsson and Mark Roden 2015-07-13
2+
Lots of work, mainly adding conversion to color.
3+
14
x3f_052: Roland Karlsson 2014-11-30
25
Running valgrind and removing some leaks and a buffer
36
overrun bug. Now you can handle several files in the same

doc/copyright.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,10 @@ BSD-style License
1919
* names of its contributors may be used to endorse or promote products
2020
* derived from this software without specific prior written permission.
2121
*
22-
* THIS SOFTWARE IS PROVIDED BY ROLAND KARLSSON ''AS IS'' AND ANY
22+
* THIS SOFTWARE IS PROVIDED BY COPYRIGHT HOLDERS ''AS IS'' AND ANY
2323
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
2424
* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
25-
* DISCLAIMED. IN NO EVENT SHALL ROLAND KARLSSON BE LIABLE FOR ANY
25+
* DISCLAIMED. IN NO EVENT SHALL COPYRIGHT HOLDERS BE LIABLE FOR ANY
2626
* DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
2727
* (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
2828
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND

doc/readme.txt

Lines changed: 55 additions & 96 deletions
Original file line numberDiff line numberDiff line change
@@ -1,139 +1,98 @@
1+
----------------------------------------------------------------
2+
X3F Tools is a library for accessing Sigma X3F raw image files
3+
----------------------------------------------------------------
14

5+
Reading of the format:
26

7+
The code understands the old format for SD9, SD10 and SD14. It also
8+
understands the new format introduced with DP1. The latter format is
9+
associated with the TRUE engines, TRUE I and TRUE II. Currently it
10+
also supports the Merrill version and up to the slightly different
11+
Quattro version of the format.
312

4-
OUT OF DATE !!!!!!!
5-
OUT OF DATE !!!!!!!
6-
OUT OF DATE !!!!!!!
7-
OUT OF DATE !!!!!!!
8-
OUT OF DATE !!!!!!!
9-
OUT OF DATE !!!!!!!
10-
11-
12-
TODO : needs serious rewriting!
13+
Converting to color images:
1314

15+
The code can convert all the above formats, except the Polaroid x530,
16+
to DNG files. Those DNG files contain color conversion data so that
17+
e.g. Lightroom can convert them to color-correct images.
1418

19+
The code can also convert to color-correct TIFF images, but out of
20+
gamut colors are not handled correctly, in particular if the input
21+
channels are clipped. The latter leads to e.g. colorful skies.
1522

1623
----------------------------------------------------------------
17-
X3F tools is a library for accessing Sigma X3F raw image files
18-
----------------------------------------------------------------
24+
Included in the library are two tools:
1925

20-
The code understands the old format for SD9, SD10 and SD14. The code
21-
also understands the new format introduced with DP1. The latter format
22-
is associated with the TRUE engines TRUE I and TRUE II. Currently the
23-
code supports up to the Quattro version of the cameras, but do only
24-
understand meta data up to Merrill.
26+
x3f_extract A tool that extracts JPEG thumbnails and raw images.
27+
See below for usage. The RAW images may also be
28+
converted to DNG or TIFF color images. Metadata and
29+
histograms over the data might also be written.
2530

26-
----------------------------------------------------------------
27-
Included in the library are two tools
28-
x3f_extract A tool that extracts JPEG thumbnail and raw images.
29-
See below for usage.
30-
x3f_io_test A tool that prints some meta data and tests that
31+
x3f_io_test A tool that prints some metadata and tests that
3132
the code is working properly. This tool is not
32-
made to be user friendly. Its mainly a test
33-
tool for developing the code. This tool can
34-
also write x3f files. They should (with this
35-
tool) be identical to the origanal file.
36-
See below for usage.
33+
made to be user friendly. It is mainly a testing
34+
tool used for development.
3735
----------------------------------------------------------------
3836

3937
----------------------------------------------------------------
4038
Executables
4139
----------------------------------------------------------------
4240

43-
It is a fair chance that you can find pre built executables on
44-
http://www.proxel.se/x3f.html.
41+
Prebuilt executables will be available at: http://www.proxel.se/x3f.html
4542

4643
----------------------------------------------------------------
4744
Building
4845
----------------------------------------------------------------
4946

50-
You are supposed to have installed gcc (and gmake) on your machine.
51-
On Windows this currently (2010) means mingw.
47+
You are supposed to have gcc, gmake and cmake installed on your
48+
machine. Currently you need to build on Linux with cross-compilation
49+
for Windows and OSX.
5250

5351
The command "make" builds the executables.
5452

55-
The makefile tries to find out what platform you are on. This might
56-
fail if your system has unexpected properties. In this case you are on
57-
your own and have to hack the makefile file.
53+
The makefile tries to find out which platform you are on. This might
54+
fail if your system has unexpected properties. In that case you are on
55+
your own and have to hack the makefile.
5856

5957
----------------------------------------------------------------
6058
Usage of the x3f_extract tool
6159
----------------------------------------------------------------
6260

63-
The tool can be used thus:
64-
65-
(1) x3f_extract -raw file.x3f
66-
(2) x3f_extract -tiff file.x3f
67-
(3) x3f_extract -tiff -gamma 2.2 [-min 0] [-max 5000] file.x3f
68-
(4) x3f_extract -jpeg file.x3f
69-
(5) x3f_extract -ppm file.x3f
70-
(6) x3f_extract -ppm -gamma 2.2 [-min 0] [-max 5000] file.x3f
71-
(7) x3f_extract -meta file.x3f
72-
73-
(1) dumps the data block of the RAW image verbatim. The original RAW
74-
data is not parsed or interpreted and is therefore for almost all
75-
aspects useless. Mainly used for analyzing a new unknown format.
76-
77-
(2) parses and interprets the RAW image data and then dumps the result
78-
as a 16 bit TIFF file (called file.x3f.tif) without doing any
79-
changes to the pixel values.
61+
You will get complete information on the switches by running
62+
x3f_extract without any switches, or e.g. with the switch -help.
8063

81-
NOTE - if you load this TIFF file in e.g. Photoshop - the result
82-
is a very dark (almost black) image. This image needs scaling and
83-
gamma coding to look good.
64+
Here are some examples:
8465

85-
NOTE - the result is not a regular RGB image - its in the X3F
86-
"color space" - so it contains faulty colors.
66+
(1) x3f_extract -denoise file.x3f
67+
This one creates the file file.dng for usage in e.g. Lightroom
68+
or Photoshop (via ACR). The file contains denoised but unconverted
69+
RAW data plus color conversion info.
8770

88-
(3) is the same as (2) but it scales the image according to
89-
a min, a max and a gamma value. If the min and max values are not
90-
given then they are estimated.
71+
(2) x3f_extract -denoise -tiff -color sRGB -crop file.x3f
72+
This one creates the file file.tif for usage in e.g. Photoshop.
73+
The file is fully converted to sRGB, but without rendering intent
74+
so you might get strange clipping.
9175

92-
NOTE - the current code does not really know how to compute
93-
the correct min and max values. Its recommended to set it yourself
94-
- a guess is that max might be between 5000 and 10000. The program
95-
writes the min and max value it finds. If you are outside the range,
96-
the value is clipped.
76+
(3) x3f_extract -unprocessed file.x3f
77+
This one creates the file file.tif with raw data. The data is
78+
linear and unscaled. So, it will generally look black and needs to
79+
be rescaled and gamma 2.2 has to be applied to it in your editor.
9780

98-
(4) dumps the embedded JPEG thumbnail verbatim as file.x3f.jpg
81+
(4) x3f_extract -meta file.x3f
82+
This one dumps metadata to file.meta
9983

100-
NOTE - this is not a JPEG of the RAW data.
101-
102-
(5) Same as (2) but output is binary 16 bit PPM
103-
104-
(6) Same as (3) but output is binary 16 bit PPM
105-
106-
(7) dumps the meta data extracted from the file.
10784

10885
----------------------------------------------------------------
10986
Usage of the x3f_io_test tool
11087
----------------------------------------------------------------
11188

112-
The tool can be used thus
89+
This tool is really only a debugging tool for development. The tool
90+
can be used thus:
11391

11492
(1) x3f_io_test file.x3f
115-
(2) x3f_io_test file.x3f -unpack
116-
(3) x3f_io_test file.x3f -write
117-
(4) x3f_io_test file.x3f -unpack -write
118-
119-
(1) reads the file and parses the main data structure of the file and
120-
then prints some info about it. NOTE - no parsing of data blocks,
121-
e.g. image blocks, is made.
122-
123-
(2) same as (1) but also parses data blocks, and then prints info again.
124-
125-
(3) Same as (1), but also writes from the unparsed version.
126-
127-
(4) Same as (2), but also writes both from the umparsed version and
128-
the parsed.
129-
130-
NOTE - detaled assembling of the blocks is not yet implemented, so
131-
the assembling is partly phony. Therefore - that oufile2 is
132-
identical does not say so much as it should.
133-
134-
NOTE - the printouts contain CAMF parameter names. The current
135-
implementation stops parsing CAMF when finding something it does
136-
not understand. That might be wrong. Therefore - parameters might
137-
be missing.
93+
Reads the file, parses the main data structure, and then prints
94+
some info about it. NOTE - no parsing of data blocks, e.g. image
95+
blocks, is done.
13896

139-
NOTE - the actual contents of the CAMF parameters is not yet parsed.
97+
(2) x3f_io_test -unpack file.x3f
98+
Same as (1), but also prints info from the parsed data blocks.

doc/todo.txt

Lines changed: 4 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -8,46 +8,23 @@ REFACTORING
88
Simple naming fix. A layer should always be called a layer.
99
The layers might be called colors after converting to RGB.
1010

11-
* Better error handling.
12-
1311

1412
--------------------------------------------------------------------
1513
CAMF AND OTHER FILE FORMAT ISSUES
1614
--------------------------------------------------------------------
1715

1816
* The CAMF matrix data has the dimensions in the description out of
19-
order, at least for older cameras. What does that mean for the
20-
stored data? Does that follow the order or the index n in the info?
17+
order, at least for older cameras and also for Quattro. What does
18+
that mean for the stored data? Does that follow the order or the
19+
index n in the info?
2120

2221
* Signed vs unsigned for CAMF matrix data ?
2322

24-
* Different names for the same property list, e.g.
25-
WhiteBalanceColorCorrection.
26-
27-
* Real lookup of matrix names, via property lists.
28-
29-
* Wrong file format for Quattro, e.g. white_balance is in other place.
30-
31-
* We cannot get max raw values correctly.
32-
33-
* Quattro probably also needs a min value.
34-
35-
* Extended types ?
36-
37-
* Double instead of float ?
3823

3924
--------------------------------------------------------------------
4025
SMALL THINGS
4126
--------------------------------------------------------------------
4227

43-
* Default write in curent directory.
44-
45-
* Define output directory.
46-
47-
* Define output file name.
48-
49-
* Write to foo.tiff.parts and then rename to foo.tiff.
50-
5128
* Logic for getting thumbs images is wrong. There might be two
5229
and they might be of same type but different size.
5330

@@ -58,53 +35,13 @@ SMALL THINGS
5835
COLOR AND CONVERSION
5936
--------------------------------------------------------------------
6037

61-
* Support older (SD14 and older) file format for color conversion.
62-
WBCorrection_<WB> and CamToXYZ_Flash.
63-
64-
* What connection space is used in TRUE engine? Older is
65-
CIE XYZ and TRUE uses some kind of RGB. We are using CIE
66-
RGB now. We also convert D50 to D65. Are those right
67-
choices?
68-
69-
* Nonlinear coding of output.
70-
71-
* How to handle the problem with clipped (overexposed) input.
72-
73-
* Scaling. Why is the image dark?
74-
75-
* Auto exposure correction?
76-
77-
* ColorModeCompensations etc
78-
79-
* Quattro ColorTemp is computed otherwise than other wb
80-
81-
* There are also more matrices for color for Quattro
82-
83-
* Contrast/(Sharpness)/Saturation
84-
85-
* Crop to image rectangle
86-
87-
* Noise!!
88-
89-
--------------------------------------------------------------------
90-
COLOR AND CONVERSION (ADVANCED)
91-
--------------------------------------------------------------------
92-
93-
* Lots of matrices with fault info? What to do?
94-
95-
* Lookup tables ?
38+
* Green cast ?
9639

9740

9841
--------------------------------------------------------------------
9942
LONG TERM
10043
--------------------------------------------------------------------
10144

102-
* Understanding important meta data
103-
104-
* Making a usable color correct image
105-
106-
* Making a usable B&W image
107-
10845

10946
--------------------------------------------------------------------
11047
DOCUMENTATION

makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ $(VENV)/.setup.touch: $(REQUIREMENTS) | $(VENV)
7474
$(VENV)/bin/pip install -r $< && touch $@
7575

7676
check: check_deps dist
77-
DIST_LOC=dist/x3f_tools-$(shell git describe --always --dirty)-$(TARGET)/bin/x3f_extract$(EXE) $(BEHAVE)
77+
DIST_LOC=dist/x3f_tools-$(shell git describe --always --dirty --tags)-$(TARGET)/bin/x3f_extract$(EXE) $(BEHAVE)
7878

7979
clean_deps:
8080
rm -rf $(VENV)

0 commit comments

Comments
 (0)