High ISO noise

From  Kubuntu breezy onwards The d70 is automatically recognized and mounted. KDE 3.5 automatically adds a desktop icon which should make windows users happy.

Kubuntu breezy pops a "Camera" desktop icon upon connection.
gphoto2 version 2.1.5 will download all photos to the current directory with 'gphoto2 -P'.

gphoto2 2.1.5 has some support for the D70,

You can capture an image with,

The intriguing summary option provides camera status, with a few known fields and many unknowns. I wish (readwrite) would mean can be set by gphoto2, but this will probably never happen.

Steve Harris sent me this patch for libgphoto 2.1.5 (cd libgphoto2-2.1.5/camlibs/ptp2; patch < libgphoto-2.1.5-nikon.patch), which  "adds name support for most of the D70 extension parameters, and renders some of them in a human readable way".
He also led me to ptpcam, which in fact can change camera settings. However it is a command line only affair, and not a particularly nice one to be honest. It seems like all the pieces are in place, but I fail to see how it is usable without a good GUI. Is there such a beast? Can you write one?

gtkam version 0.1.12 compiled and produced an executable and a gimp plug-in. The plug-in adds "Capture from camera" and "Download from camera" entries to the gimp Acquire submenu. Both are useless. Download presenting a garbled 120x160 image. Capture activates the camera and exists with an obscure error message. Manually detecting the camera each activation is no fun either.

EXIF tags:

• Reading
  

The best program to read D70 NEF and JPG metadata is Phil Harvey's ExifTool. All the data discussed in the D70 manual (and then some) is recognized (I know that for a fact because I sent the initial data to Phil ...). The full set is quite large so it is useful to create a .ExifTool_config file to view just a subset (Look for "Shortcut Tags" in Phil's page). The latest version can write and transfer tags as well. Minor drawback - it is written in Perl.

Raw conversion via dcraw discards EXIF tags.  neftags2jpg transfers them back to the jpg.  neftags2jpg uses exiv2 (get version 0.7 or newer).

After compiling and installing, simply call

KDE Konqueror shows NEF thumbnails automatically, since they are in fact TIFF files, declaring the small 120x160 thumbnail as the "main image".  Remember to increase file size in the "Previews and Meta data" Konqueror configuration to at least 6MB.

gqview is a capable and fairly speedy viewer. To view NEF thumbnails, click Edit/Options.../Filtering, double click on the tiff entry and add ;.nef to the list.
gqview, which uses the GTK library to load images, finds the first (smaller 120x160) thumbnail but knows nothing of the second, larger one. Here is an appalling patch for version 1.5.X to remedy the situation, which requires a working nefextract in you path first. You want jpegtran in your path as well to enable auto rotation (there is a good chance it is already installed for your system).  Right click and choose (Edit -> In the Gimp...) to complete a reasonable single photo workflow.

Goggle Picasa has a working Linux version which (slowly) handles raw images, but may be better than other free solutions at this point.

If you are interested in improving your photographic skills (and in some cases the images), it is best to shoot in RAW mode.  However this shifts more of the responsibility in developing prints into your hands, a decision not be taken lightly.

For some time Dave Coffin's dcraw has been the only option. Dave has made an amazing job reverse engineering every major (and some non major) raw formats on the planet. In addition to adding new cameras at a phenomenal rate, Dave is improving the usability of dcraw by adding automatic rotation, color management, more advanced interpolation methods and so on. Here is my modified V8.40 (October 21 2006) version which contains some minor changes used by the gimp plugin (see below) which add reading tone curves and thumbnails.

 UFRaw (0.9.1) has a standalone, 16 bit version for batch processing.

Bibble 4.x Has a Linux version with D70 support. You can download a time limited trial version and check it out.

I bought Bibble Pro in April 2006 and has been using it since. I have mixed feelings about Bibble. The workflow is well thought of and it is full featured while keeping reasonably fast. However, the main reason I got bibble was to be able to print but this is broken in several ways and even though it is a professional, paid for software in Linux space no one will hear your screams. I am not always happy with the resulting image but I may be expecting too much or fretting about non issues. The Linux version is also prone to crashes, hangs, and getting stuck while happily using 100% of the  CPU.

Mark from dpreview gave the link to through-the-lens RawViewer, a Java/dcraw based viewer and conversion tool.

Personal note: I don't really care for Java. In a true java spirit the user interface has some oddities - The directories are sorted in a semi random order, hidden directories are shown by default, "camera curve" has to be selected for any new file even though it appears as the default menu choice, the jpeg in portrait orientation came out as landscape, if you don't select an output directory the program goes through the motions, prints a large friendly DONE at the end, but does nothing. I did not find any "Auto exposure" button, but I might have missed it.

• Free for Linux

At the start of 2004 there were practically no good options for RAW processing on Linux, free or not. The situation has improved to a point where you can choose between one commercial product and several free options, three of them related gimp plugins. The first is the original PTJ's RawPhoto. The second is my fork (RougePhoto 1.39e, October 21 2006), which is substantially different by now. The third is Udi Fuchs UFRaw, another fork by a D70 enthusiastic. I guess D70 attracts more geeks than other models ... UFRaw, now in version 0.9.1, is undoubtedly the leader in open source raw conversion tools.
 
Please use RougePhoto with at least gimp 2.2 and gcc 3.3. Don't send me emails about problems with earlier versions.

Note: You must use my modified dcraw (above) with RougePhoto. In the future I might integrate it in, but preliminary tests show no speed improvement, so it is very low on my priority list.

Note: If you are getting images with strange colors, try running with  'env LANG=en_US gimp' (Assuming you run under a different setting). I hope to eliminate this workaround when I figure out what is going on.
The locale problem is hopefully fixed in the latest version. Please report if it persists for you.

Note: The plugin will load any file supported by dcraw, which are many. However, note that for your colors to be closer to reality you need to find the appropriate input ICC profile for your camera. See below.

I tested RougePhoto with the following formats

• CRW (Canon 300D and 10D)
  
• MRW (Minolta)
  
• NEF  (D100 and D70)
• ORF (Olympus E-300)

I would appreciate example files from other formats if you have them.

Note: EXIF tags support is available for D70 when neftags2jpg is installed. However, gimp uses libexif, and version 0.6.9 (the latest available) handling of NEF makernote is broken. For now I advice removing the test for Nikon in libexif/exif-data.c. It is far from obvious how to do this in Debian - see here for details.

RougePhoto now contains some color management capability via Little CMS. Read Udi's excellent Color management page. To setup RougePhoto, download Nkx_D70_4_0739_02_308.icm from Udi's site and place it in ~/.gimp-2.2/profiles/input. Copy (Or soft link) any output profiles to ~/.gimp-2.2/profiles. See the README file for the gimprc entry needed to complete the setup. I have made only very limited testing using images from other cameras, but duplicating the above with the appropriate files should work. Some 10D owners reported they had no success with the 10D profile in Udi's site. I suggest using either Fred Miranda's or Bibble profiles for the 10D (Can someone find out if there is an official Canon profile?).

The recent version of dcraw (7.30 and up) contains "conversion matrices" from camera RGB to sRGB prepared by Thomas Knoll. If you can't set an input profile, those provide much better output. Great addition, Dave!  As of version 1.36e, RougePhoto will use those automatically if it can't find an input profile. An override toggle is provided in the Setup tab.
Below is a comparison for a Canon 10D image.

Canon thumbnail

No Color Management

Bibble 10d profile

dcraw sRGB

Unfortunately the D70 conversion is not as good results in a "washed out" image. Using the Nikon profile is the best bet at this stage for D70 nef's.
Plugin Features:

• Spot WB, White and Black Point.
• Gamma, Exposure and Black level.
• Auto Exposure/Black point (with separate exposure and Color Clipping threshold).
  
• Color Auto Level.
  
• Saturation & Shadows adjustments
  
• B&W mode.
  

• Visual display of overexposure,underexposure, and color clipping.
• Zooming, Panning and Resizing of preview (thanks to Dov Grobgeld GtkImageViewer)
• Intensity and Color channel histograms.
• Manual & Automatic orientation (when supported by camera).
• Plugin title shows image name and camera make/model.

• User control of White/Black/Color Clip cutoff%.
• Choice of saturation method. Either linear or the one from Udi Fuchs  UFRaw. I don't use saturation in raw conversion, but Udi says it is an improvement.
• Gamma and Log tone curves
• Camera Tone Curve. dcraw loading code taken from Udi Fuchs  UFRaw.  (Again only for D70. I will add more if anyone sends me a dcraw patch to read tone curves from other raw formats)
• Load-able Tone Curves. Unpack curves (Dec 07) into your gimp curves directory (typically ~/.gimp-2.2/curves) and they should appear in the tone curve selection menu.
• Automatic selection of input color profile. Menu of Destination color profiles. Download Nkx_D70_4_0739_02_308.icm from Udi's site and place it in ~/.gimp-2.2/profiles/input. Place (Or soft link) any output profiles to ~/.gimp-2.2/profiles. See the README file for the gimprc entry needed to complete the setup.
  

• Image comment contains RougePhoto conversion parameters for later reference.
• Fast startup.  (2-3 seconds on my 1800XP Athlon using the default setup).
• Conversion takes approximately 12-13 seconds on same machine.
  
• EXIF support for D70 via neftags2jpg.
• Some customization options via gimprc.
  
• Gimp 2.2 ready.
• Support for Gimp 2.2 thumbnail API. D70 now uses small 120x160 thumbnails for speed.
  

Raw Plugin Screen Shot (KDE 3.3.1 - Keramik, GTK 2.4.4 - Aquativo-1.5)

High ISO noise:

At higher ISO's, CCD noise is quite visible. It seems Nikon Capture (and other RAW converters)  automatically apply noise reduction in higher ISO's, but at the moment Linux only users have to apply noise reduction using Gimp.

Below is a 100% crop from a photo shot at ISO 800. Below are the results from the professional Noise Ninja (Bill's Box Only), Gimp Despeckle plugin, Dcam Noise 2, GREYCstoration, Film Grain Remover, Noise Remover plugin, and the slightly improved noise remover described later.

One problem in evaluation is that it is easy to spot artifacts, but hard to measure the amount of noise reduction. It is impossible to figure out how to apply the "same amount" of reduction when each method offers totally different, and typically cryptic controls.
For the moment I will ignore this issue, unless it is obvious (see Grain remover below).

original
Noise Ninja	Despeckle (default)
Noise remove	Noise Remove + CC
Film grain remove	Dcam2 Noise
GREYCstoration

Below are the difference images (via Grain Extract layer mode) to the original and a closer crop for each.
Despeckle, beside being horrendously slow, exhibits strong color dependent patterns. Noise-remover shows smearing, halos and large color loss, especially in the blue channel. Grain remover looks great, unfortunately this is because in it's default setting it removes little noise. In higher settings it exhibits considerable smearing and lose of detail.
I recently got an email from Peter Chiocchetti pointing me to Dcam Noise 2. This seem to be an impressive plugin with many (perhaps too many?) controls. However by using the defaults and just adjusting "Filter Threshold" I got a very good result which seem to have less artifacts than Noise Ninja and still reducing the noise nicely.
Niels Kristian Bech Jensen sent me the link to GREYCstoration, another heavy duty plugin. Just like Dcam Noise 2, it has various cryptic controls and no advice on how to use them, so the chance of doing justice to both are low. GREYC does a nice job of removing noise while preserving color and detail. However it is much slower, turning even exploration via the preview quite painful.

Noise Ninja
Despeckle
Noise Remove
Noise Remove + CC
Film Grain Remove
Dcam Noise 2
GREYCstoration

The small, non scientific change I made to noise-remover is to simply disallow pixel changes greater than TH (say 20). The result is obviously more noisy, but does not exhibits strong color patterns. The price is high on another front - I was reluctant to write a plugin for what seems a simple task, however iterating over an image is way over script-fu ability, and even python-fu, which I used, needs to be augmented with heavy machinery in the form of numarray. You can get my python-fu script here.
With the Dcam and GREYC in my Gimp menu I am happy to say Linux users are not at a disadvantage anymore - and as usual - the price is just right.

While browsing yesterday I saw that the Vosonic X's Super drive 6230 supports D70 RAW files, and after a brief googling session bought one on impulse over the internet. The unit has a built in CF card reader and a 2.5" screen for viewing (+ zooming), can play audio and video and has an upgradeable hard drive (I got the 80G version).

Unfortunately all the online complaints turn out to be true. The unit is an unattractive silver plastic brick, the buttons feel as if they would last only a short time, the sound quality is marginal and the limited video codec support mean I will have to convert any video I wish to view them on the tiny screen. The user interface is clunky, user manual outdated and useless, and the list goes on.

While you can indeed view NEF files, EXIF info is not available. It is there for JPG's but thanks (again) to Nikon failing to fix this bug even in Firmware 2.0, ISO is displayed as 0 .

On the other hand the unit has an upgradeable drive and batteries, USB 2.0 is fast (backward compatible with 1.1 as well), is Linux compatible by choice and not by hacking, and may improve with upcoming firmware updates. I hope I will have some good things to say after using the unit in the field.

(Oct 2006)  The  80 gig on the X drive has filled up nicely. On the whole it was a good buy, as the unit travels well and still fully functional. Vosonic keeps supporting the model by releasing firmware updates and fixing small issues on the way. My music now resides on an ipod video, so the marginal audio quality is not an issue anymore.

As you can see, my English language & graphic design skills leave a lot to be desired. If you are experienced in either and would like to help, I will appreciate it greatly.

Local links

neftags2jpg - Transfer EXIF tags from NEF to JPG.
nefextract - Extract full size preview from a NEF file.
modified dcraw (October 21 2006) - (required by plugin)
RougePhoto (1.39e October 21 2006) - Gimp 2.2 plugin.
Tone curves (Dec 07) - Tone curves for RougePhoto plugin.
noise-remover + CC plugin (Dec 02)

External links

ExifTool - Read EXIF data.

- gimp raw plugin.

RawPhoto - PTJ gimp raw plugin.

ptpcam - control camera via USB

libgphoto2 patch









Dcam Noise 2

Please send comments, suggestions and feedback to  jheled@.com