Libgourou v0.5

Sunday, 19 December 2021
|
Écrit par
Grégory Soutadé

Reminder : Libgourou is a free ADEPT protocol implementation (from Adobe) that helps download ACSM files from Linux.

It's Christmas soon. A time of heavy natural resources consumption, in all domains, because tradition and technological progress makes earth burning a bit more again and again. I suggest you to offer culture more than stupid and useless things.

And, what a good news, I have a gift for you ! Libgourou in version 0.5. It doesn't look like great as it's not the v1.0, but it's a big update. First versions were mainly bugfixes (plus PDF support) and I would like to tkank everyone who reported all that, more or less, stupid bugs. Reporting is not funny, but very useful for me and everybody.

So, my first intention when I created libgourou was not to support DRM removal but I saw a lot of people buying PDF (while I was focused on ePub). Using an eReader for reading PDF is not the best solution, big colored screens are so much better. After a long work in both libgourou and uPDFParser, you can now use the new adept_remove utility to remove DRM form ePub AND PDF ! Another good thing is the add of anonymous account support (no need to create or use your account from adobe.com). I recommend to use anonymous account only with a DRM removal software (like adept_remove), because the book will not be linked to your account and in case of failure, you'll have to buy/loan it once again.

I hope you'll enjoy this release. You can retrieve source code here or directly download pre compiled binaries (for Debian testing) here.

See you in 2022 !

#
1
De
Milian Wolff
, le
07 February 2022 05:02
Hey there,

I was hoping to use your tool to download a book I rented from my library. I am using gourou via the archlinux AUR package in version 0.5. Sadly, the acsmdownloader is just sitting there in a busy loop consuming one CPU core but not doing any apparent progress in actually obtaining the ePUB file.

GDB shows me that the following background thread is at fault:

```
Thread 2 (Thread 0x7ff03d00d640 (LWP 11569) "Thread (pooled)"):
#0 0x00007ff03f9c18f1 in g_main_context_prepare () from /usr/lib/libglib-2.0.so.0
#1 0x00007ff03fa17ed6 in ?? () from /usr/lib/libglib-2.0.so.0
#2 0x00007ff03f9bf485 in g_main_context_iteration () from /usr/lib/libglib-2.0.so.0
#3 0x00007ff040e8b4a8 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQt5Core.so.5
#4 0x000055fa9dbc0307 in DRMProcessorClientImpl::sendHTTPRequest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*) ()
#5 0x00007ff040874d1a in gourou::DRMProcessor::sendRequest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char const*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >*) () from /usr/lib/libgourou.so
#6 0x00007ff040878403 in gourou::DRMProcessor::download(gourou::FulfillmentItem*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) () from /usr/lib/libgourou.so
#7 0x000055fa9dbba14d in ACSMDownloader::run() ()
#8 0x00007ff040c9caa2 in ?? () from /usr/lib/libQt5Core.so.5
#9 0x00007ff040c9888c in ?? () from /usr/lib/libQt5Core.so.5
#10 0x00007ff0402f4259 in start_thread () from /usr/lib/libpthread.so.0
#11 0x00007ff04040a5e3 in clone () from /usr/lib/libc.so.6
```

Any idea what the issue might be here?

Thank you so much for working on this tool!
Répondre
Auteur :


e-mail* :


Le commentaire :


#
2
De
Greg
, le
07 February 2022 07:02
Hello,

It seems HTTP request doesn't send any response. Can you use verbose mode (-v -v -v) and eventually try to download manually the file to see if there is any problem ?
Répondre
Auteur :


e-mail* :


Le commentaire :


#
3
De
Milian Wolff
, le
08 February 2022 10:02
I'm on a flaky internet connection here, so maybe that's the fault. With verbose mode I see the last output as:

```
Send request to https://dp1.onleihe.de/downloadprovider/epub/<...>.epub
```

With wget I can download that URL seemingly, albeit slowly. I'll see if I can make it work that way, thanks!
Répondre
Auteur :


e-mail* :


Le commentaire :


#
4
De
Milian Wolff
, le
08 February 2022 11:02
So now that I have the epub, I cannot read it. Do I have to remove the DRM or is there another option? I mean I'm allowed to read it, I loaned the book for two weeks, but it seems basically impossible to do that on linux currently. even trying DRM removal for a test doesn't work:

```
Parse activation file .adept/activation.xml
Exception code : 0x5008
Message : Zip error, no file META-INF/rights.xml, No such file
File : drmprocessorclientimpl.cpp:439
```

so I guess I'll have to give up now, or do you know a way forward?
Répondre
Auteur :


e-mail* :


Le commentaire :


#
5
De
Greg
, le
08 February 2022 13:02
I think the best way here is to try to remove DRM from your ebook. In order to do that, you have to create a file named META-INF/rights.xml inside epub (which is a zip file). Content is

<?xml version="1.0"?>
<adept:rights xmlns:adept="http://ns.adobe.com/adept">
<licenseToken xmlns="http://ns.adobe.com/adept">
...
</licenseToken>
</adept:rights>

You can retrieve information of licenseToken using verbose mode (-v -v -v). This is normally done by libgourou once file is dodwnloaded.
Répondre
Auteur :


e-mail* :


Le commentaire :


#
6
De
Milian Wolff
, le
08 February 2022 14:02
Thanks!

With that done, `adept_remove` now hangs here:

```
(gdb) bt
#0 0x00007fe9224f24f5 in ?? () from /usr/lib/libz.so.1
#1 0x00007fe9224f9546 in inflate () from /usr/lib/libz.so.1
#2 0x000056079907314b in DRMProcessorClientImpl::inflate(gourou::ByteArray&, gourou::ByteArray&, int) ()
#3 0x00007fe9224ca7ba in gourou::DRMProcessor::removeEPubDRM(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /usr/lib/libgourou.so
#4 0x000056079906e8cb in ADEPTRemove::run() ()
#5 0x00007fe9228f0aa2 in ?? () from /usr/lib/libQt5Core.so.5
#6 0x00007fe9228ec88c in ?? () from /usr/lib/libQt5Core.so.5
#7 0x00007fe921f48259 in start_thread () from /usr/lib/libpthread.so.0
#8 0x00007fe92205e5e3 in clone () from /usr/lib/libc.so.6
```

The CLI output it showed was:

```
Parse activation file .adept/activation.xml
Decrypted : ... hex chars ...
Encrypted file OEBPS/... # multiple times
```

I see no disk activity nor any memory consumption increase, which makes me wonder what that code is decompressing here...

So I guess there's still something amiss?
Répondre
Auteur :


e-mail* :


Le commentaire :


#
7
De
Milian Wolff
, le
08 February 2022 14:02
I have now compiled the code myself and added debug statements, it seems like the inner inflate loop has a bug, see:

appended: 89800, 44537 | -5
appended: 89800, 89800 | -5

the second line is then repeated endlessly. It seems the issue is that the buffer is too small to hold the decompressed data? We then get stuck in Z_BUF_ERROR ...
Répondre
Auteur :


e-mail* :


Le commentaire :


#
8
De
Greg
, le
08 February 2022 14:02
Can you replace Z_FINISH flag by Z_NO_FLUSH into DRMProcessorClientImpl::inflate() ?
Répondre
Auteur :


e-mail* :


Le commentaire :


#
10
De
Milian Wolff
, le
08 February 2022 15:02
Aha, that seems to help with the PNG file. But there's still that one JPG file which is triggering a different problem I guess:

Exception code : 0x5008
Message : Inflate error, code data error, msg invalid literal/length code
File : drmprocessorclientimpl.cpp:528

Should the code maybe handle Z_DATA_ERROR gracefully and skip such files?
Répondre
Auteur :


e-mail* :


Le commentaire :


#
9
De
Milian Wolff
, le
08 February 2022 14:02
The unhandled case happens when `(ret == Z_BUF_ERROR && dataSize == infstream.avail_out)`, this then will loop endlessly. In my case, it's a PNG file. If I skip that one manually I later run into a `Z_DATA_ERROR` for a JPG file. Skipping that one too, it seems to works.

But I wonder why that happens - is the ePUB source corrupted during download which then triggers these inflate issues? I even tried to allocate `UINT_MAX` sized buffers to no avail, I think something is really broken with the two PNG and JPG files in my source.

But once again - thank you very much for working on this tool suite.
Répondre
Auteur :


e-mail* :


Le commentaire :


#
11
De
Milian Wolff
, le
10 February 2022 14:02
Hey again Gregory,

I had a look at the original download problem I reported to you. That is an issue with your code - you are busy polling instead of using an QEventLoop and asynchronous code. The following is much better (zero CPU load basically) and also gives some progress report to indicate that it is doing something (albeit slowly for me):

```
QEventLoop loop;
QObject::connect(reply, &QNetworkReply::finished, &loop, &QEventLoop::quit);
QObject::connect(reply, &QNetworkReply::errorOccurred, &loop, &QEventLoop::quit);
QObject::connect(reply, &QNetworkReply::downloadProgress, &loop, [](qint64 bytesReceived, qint64 bytesTotal) {
std::cout << "dl progress " << bytesReceived << "\t" << bytesTotal << '\n';
});
loop.exec();
```

Use that instead of your while loop with process events. Consider the above as code in the public domain - I don't need any attribution.

Cheers
Répondre
Auteur :


e-mail* :


Le commentaire :


#
12
De
Greg
, le
23 February 2022 06:02
Integrated in v0.5.2. Thanks for your contribution !
Répondre
Auteur :


e-mail* :


Le commentaire :


Auteur :


e-mail* :


Le commentaire :




* Seulement pour être notifié d'une réponse à cet article
* Only for email notification