Addition of error raise in initialize causes unexpected behavior

[optiontech]

I was using the previous version (0.2.4) with fantastic results. I updated to 0.2.5 today and a number of my tests started failing. Looking at the code, it appears that this is because the initialize is now raising errors in addition to sending them to the import_failed callback. I'm having trouble understanding the reasoning behind doing both. I would think that sending the errors to the import_failed callback so that my app can handle them (which is what I'm doing) would be sufficient. Am I possibly missing some other implication here?

[gnapse]

You're right. I made the change knowingly of the implications, but I intended to make it optional. There have been users that prefer the operation to fail with an exception if the failure is total, and not just while processing rows.

Also now the gem includes a transactional mode, that when activated, makes the importer not to catch the exception that caused the transaction failure, so it seemed more natural than when the failure of importing is fatal, to expose the error. This is the case with the errors that are sent to import_failed (the file is unreadable, or does not have the expected tabular structure, etc.)

However, I understand your situation. That's why I said I would make this optional, but I haven't yet. Like a config parameter or something, and defaulting to not raising the exception, which was the original behavior. I would love your feedback on this matter.

[optiontech]

Ok. I can see wanting to have total failures short circuit the processing (faster response). That makes total sense to me. Would it be possible for active_import to determine if the error is a total failure or not? Then maybe have a separate file_failed callback for those types of errors? IMHO, it would be better to handle all the errors with callbacks rather than having some of them handled with callbacks and some with rescues (but maybe that's just me).

Also, I'm not sure I completely understand the purpose of the transactional mode. Can you explain that a bit? Perhaps that will help me understand the need for raising the error a bit more.

[gnapse]

Transactional mode encloses the whole import process inside a database transaction. Then, if a failure occurs during the import process, even a single row failing, the importer fails, and the exception causing that failure reaches the transaction block, causing it to rollback.

In any case, I totally agree with you, that the default behavior should be the original one, and I should leave the new behavior as optional perhaps. However, I should give it more thought before deciding how to achieve this.

[optiontech]

Cool. I certainly like the transactional mode (will definitely be using that). Looking forward to what you come up with for the error handling!