ImageEn for Delphi and C++ Builder ImageEn for Delphi and C++ Builder

 

ImageEn Forum
Profile    Join    Active Topics    Forum FAQ    Search this forumSearch
Forum membership is Free!  Click Join to sign-up
Username:
Password:
Save Password
Forgot your Password?

 All Forums
 ImageEn Library for Delphi, C++ and .Net
 ImageEn and IEvolution Support Forum
 ParamsFromFile(Path) extremely slow
 New Topic  Reply to Topic
Author Previous Topic Topic Next Topic  

Uwe

284 Posts

Posted - Jan 24 2013 :  10:17:38  Show Profile  Reply
Hi Fabrizio and Nigel

D2010 Pro, Vista 32bit, latest ImageEN

I just played around with images from a Sony Cybershot DSC-RX1 camera and stumbled across a really strange behavior. When I use

{...}
ImageEnVect1.IO.ParamsFromFile(Path);
with ImageEnVect1.IO.Params do
begin
if EXIF_HasEXIFData = True then
begin
{...}

reading EXIF_ExifImageWidth, EXIF_ExifImageHeight and EXIF_XResolution is extremely slow. For an image taken with the above mentioned camera it takes about 30 secs to retrieve the data, while other Sony ARW files are read in an instant.

The same slow down can be observed with XPM files by the way, and to a lesser extent also with XBM, SVG and PICT files.

Can you guys please check as this is a show stopper for me. A sample image can be found here:

http://www.imaging-resource.com/PRODS/sony-rx1/RX1INBI00800.ARW.HTM

Thanks in advance
Uwe

w2m

USA
1990 Posts

Posted - Jan 24 2013 :  11:41:01  Show Profile  Reply
I downloaded your ARW file and ran this code:
procedure TForm1.Button1Click(Sender: TObject);
begin
  Screen.Cursor := crHourGlass;
  try
    ImageEnView1.IO.LoadFromFile(DesktopFolder + 'RX1INBI00800.ARW');
    //ImageEnView1.IO.ParamsFromFile(DesktopFolder + 'RX1INBI00800.ARW');
    if ImageEnView1.IO.Aborting then
       ShowMessage('Can not open ARW files');
  finally
    Screen.Cursor := crDefault;
  end;
    if ImageEnView1.IO.Params.EXIF_HasEXIFData = True then
    begin
      ShowMessage('Image Width: ' + IntegerToString(ImageEnView1.IO.Params.EXIF_ExifImageWidth));
      ShowMessage('Image Height: ' + IntegerToString(ImageEnView1.IO.Params.EXIF_ExifImageHeight));
      ShowMessage('Image XResolution: ' + FloatToStr(ImageEnView1.IO.Params.EXIF_XResolution))
    end;
end;


When this code is executed, ImageEnView1.IO.Aborting is True so I see the error message. I do not know if the image is invalid or if ImageENIO does not support ARW files. The documentation does not list ARW files anywhere and the only the Sony DSC-F828 camera raw files are listed as supported.

William Miller
Email: w2m@frontiernet.net
EBook: http://www.imageen.com/ebook/
Apprehend: http://www.frontiernet.net/~w2m/index.html
Go to Top of Page

Uwe

284 Posts

Posted - Jan 24 2013 :  12:32:59  Show Profile  Reply
quote:
I do not know if the image is invalid or if ImageENIO does not support ARW files.

William, I tested several images of this camera from different sources and they all show the same weird behavior. It's the only Sony camera I know of that produces such images, and I suspect that Sony changed their file format again and meddled with the EXIF markers. ImageEnIO does support ARW files - I've been working with them for many years. Remove the ImageEnView1.IO.Aborting part from your code and you will see that the values are retrieved after about half a minute.

Thanks for testing by the way. Could you try with an XPM file as well, please?

Uwe
Go to Top of Page

w2m

USA
1990 Posts

Posted - Jan 24 2013 :  13:33:21  Show Profile  Reply
Uwe:

I cannot get it to load... I can not do anything with the file. Even IO.Params.FileType is 0... I guess you will have to wait for Fabrizio to respond.

William Miller
Go to Top of Page

Uwe

284 Posts

Posted - Jan 24 2013 :  13:35:40  Show Profile  Reply
William, you'll need dcrawlib.dll to display the file. You know that, right?

Out of curiosity I removed the

IEFileFormatRemove(ioRAW);
IEAddExtIOPlugIn('dcrawlib.dll');


calls from my main form's OnCreate event now, and all of a sudden reading the EXIF data is much faster. So the problem must have to do with dcrawlib.dll . Just to make it clearer, let me repeat one more time what I'm doing:

-- Get the path of a file WITHOUT actually opening the image
-- Read EXIF data via ImageEnVect1.IO.ParamsFromFile(Path)

When dcrawlib.dll is linked in, retrieval of the EXIF data works, but takes almost half a minute. Not using dcrawlib.dll reduces read time for the EXIF data to a fraction of a second.

I'm wondering now if the slow read of XPM images is the result of linking imagelib.dll . I'll check.

EDIT:
Yes, the IEAddExtIOPlugIn('imagemagick.dll') call is responsible for the slow retrieval of data from an XPM image.


Uwe
Go to Top of Page

w2m

USA
1990 Posts

Posted - Jan 24 2013 :  14:20:34  Show Profile  Reply
With:
IEFileFormatRemove(ioRAW);
IEAddExtIOPlugIn('dcrawlib.dll');
IO.Params is filled in about 25 seconds.

Without the plugin the IO.Params is empty.

William Miller
Go to Top of Page

Uwe

284 Posts

Posted - Jan 24 2013 :  14:31:20  Show Profile  Reply
Hi William

Good to know that you experience the same delay that I do. Roughly 30 seconds to retrieve three EXIF tags. Seems that Fabrizio has to take a closer look.

quote:
Without the plugin the IO.Params is empty.

Yes, sorry ... I forgot that I have some alternative code which runs if EXIF_HasEXIFData = False. It calls ImageEnVect1.IO.ParamsFromFileFormat and fills the missing data.

Thanks again
Uwe
Go to Top of Page

Uwe

284 Posts

Posted - Jan 26 2013 :  13:35:29  Show Profile  Reply
Okay, I did some additional testing now and it turns out that not only is the retrieval of certain EXIF data extremely slow, but on top of that the color rendition of some images which were developed by DCRAW and handled by ImageEN afterwards is incorrect. Wrong colors are shown for quite a few cameras by the way, which are all officially supported by DCRAW. These are:

Sony DSC-RX1 (as mentioned in my initial post)
Sony SLT-A99V

Panasonic DMC-FZ200
Panasonic DMC-G5
Panasonic DMC-GH3
Panasonic DMC-LX7

Fabrizio and Nigel, can you please address these issues? There seems to be something wrong with the ParamFromFile call as well as ImageEN's interpretation of sensor data coming from DCRAW.

Thanks!

Uwe

Go to Top of Page

fab

1310 Posts

Posted - Jan 28 2013 :  00:20:40  Show Profile  Reply
Uwe,
this seems a file caching problem. I am trying to fix it.
In the meaning time you can fix it opening imageenio.pas and locate IEPlgInREAD procedure. Inside it look for:

Stream := TIEBufferedReadStream.Create(InputStream, 1024*1024, iegUseRelativeStreams);

and change to

Stream := TIEBufferedReadStream.Create(InputStream, 32*1024*1024, iegUseRelativeStreams);

or (reducing the loading buffer to 16 bytes!!):

Stream := TIEBufferedReadStream.Create(InputStream, 16, iegUseRelativeStreams);


Please wait for a better fix.
Go to Top of Page

Uwe

284 Posts

Posted - Jan 28 2013 :  09:51:44  Show Profile  Reply
Hi Fabrizio

Both your patches have an interesting effect: they speed up loading EXIF data from images taken with the Sony DSC-RX1 (although it's still quite slow), but they slow down the reading of EXIF data from almost every other Sony camera (and only that brand!) significantly. So this is not the solution, I'm afraid.

The XPM file format continues to be a problem, too, and loading both image and EXIF data is dead slow. I'll wait for your final fix in the next release.

The mismatching color issue hasn't been solved yet and I assume that you either overlooked it in my previous post or that you are working on it. Would you please confirm?

Thx
Uwe


PS: Can you please change the BPL directory path in your installer to the standard one?
Go to Top of Page

fab

1310 Posts

Posted - Feb 01 2013 :  09:11:51  Show Profile  Reply
Hi Uwe,
many thanks for your tests.
I tried two Sony raws (MRW and ARW), but the parameters loading is almost instantaneous.
Please could you provide a link?

quote:
The mismatching color issue hasn't been solved yet and I assume that you either overlooked it in my previous post or that you are working on it. Would you please confirm?


I'm sorry, I don't confirm. Is there a topic in this forum?
Go to Top of Page

Uwe

284 Posts

Posted - Feb 01 2013 :  13:35:22  Show Profile  Reply
Hi Fabrizio

Just to make sure that we are on the same page...

a) imageenio.pas without your patches applied

If I try to retrieve EXIF_ExifImageWidth, EXIF_ExifImageHeight and EXIF_XResolution from an image taken with the Sony Cybershot DSC-RX1 camera, reading the parameters is extremely slow. But only for ARW files from this particular camera, while files from all other Sony cameras are read instantaneously. William has confirmed this behavior already.

Here is a sample file: http://www.imaging-resource.com/PRODS/sony-rx1/RX1INBI00800.ARW.HTM

XPM files and to a lesser extent also XBM, SVG and PICT files are also really slow when it comes to reading EXIF_ExifImageWidth, EXIF_ExifImageHeight and EXIF_XResolution.


b) imageenio.pas with your patches applied

Reading EXIF_ExifImageWidth, EXIF_ExifImageHeight and EXIF_XResolution from ARW files taken with the Sony DSC-RX1 camera is faster (but still kind of slow) now, while reading the parameters from ARW files taken with any other Sony camera is really slow all of a sudden.

XPM, XBM, SVG and PICT files are still slow.


Would you please check again? If you can't confirm this behavior I will prepare a demo for you.

------------------------------------------------------------------

Regarding the color mismatch when displaying files from certain Panasonic and Sony cameras, please see 4 posts above. The cameras that show wrong colors are:

Sony DSC-RX1
Sony SLT-A99V

Panasonic DMC-FZ200
Panasonic DMC-G5
Panasonic DMC-GH3
Panasonic DMC-LX7

All these cameras are supported by DCRAW.

Thanks
Uwe
Go to Top of Page

fab

1310 Posts

Posted - Feb 01 2013 :  23:44:58  Show Profile  Reply
XPM, XBM, SVG and PICT are slow because they are not directly supported by ImageEn (maybe use load using the no-more-supported imagemagic plugin), so the full image is loaded every time.

About the wrong colors, please let me know the links for tests.
Go to Top of Page

Uwe

284 Posts

Posted - Feb 02 2013 :  10:58:22  Show Profile  Reply
Okay, here are links to sample images. Scroll down each page to see the camera RAW samples.

Sony:
http://www.imaging-resource.com/PRODS/sony-rx1/RX1INBI00800.ARW.HTM
http://www.photographyblog.com/reviews/sony_a99_review/sample_images/

Panasonic:
http://www.photographyblog.com/reviews/panasonic_lumix_dmc_fz200_review/sample_images/
http://www.photographyblog.com/reviews/panasonic_lumix_g5_review/sample_images/
http://www.photographyblog.com/reviews/panasonic_lumix_dmc_gh3_review/sample_images/
http://www.photographyblog.com/reviews/panasonic_lumix_dmc_lx7_review/sample_images/


Regarding the Imagemagick plugin, do you see a chance to ever update that one to support the latest version of Imagemagick?
Go to Top of Page

xequte

38613 Posts

Posted - Feb 03 2013 :  00:14:50  Show Profile  Reply
Hi Uwe

Unfortunately it is not a simple process to update the ImageMagick plug-in, but it is on the to-do list.



Nigel
Xequte Software
www.xequte.com
nigel@xequte.com
Go to Top of Page

Uwe

284 Posts

Posted - Feb 03 2013 :  12:26:56  Show Profile  Reply
Hi Nigel

Yeah, I know that it's a nightmare. Tried it once but the Imagemagick people kept changing their API, so I gave up. If I recall correctly, there once was a project called DelphiMagick by Nils Haeck, but I don't know what happened to it.

I'll add a request, maybe there's enough interest for an update.

Thanks
Uwe
Go to Top of Page

fab

1310 Posts

Posted - Feb 06 2013 :  10:38:13  Show Profile  Reply
I just tried today, and it seems that using dcrawlib 1.454 fixes some color problems. It is now available from the Registered Users page.
Go to Top of Page

Uwe

284 Posts

Posted - Feb 07 2013 :  11:13:05  Show Profile  Reply
EDIT:

Fabrizio, I just tested the new dcrawlib.dll, and the color mismatches have disappeared. Loading EXIF data from the Sony DSC-RX1 is much faster now, but still noticeably slower than from other Sony cameras.

Thx
Uwe
Go to Top of Page
  Previous Topic Topic Next Topic  
 New Topic  Reply to Topic
Jump To: