Welcome, Guest
Python Scripts for ComicRack

TOPIC: FromDucks - Scrape from I.N.D.U.C.K.S. (V 2.11)

Re: FromDucks - Scrape from I.N.D.U.C.K.S. (V 1.1b) 6 years 9 months ago #11727

  • mizio66
  • mizio66's Avatar
  • Offline
  • Platinum Boarder
  • Started reading comics at 4... and still counting!
  • Posts: 451
  • Thank you received: 143
  • Karma: 67
Updated in the first post.

***** For all those interested *****
This is a beta. 600 is helping kindly to bugfix... if you are interested... let me know
***** For all those interested *****
The administrator has disabled public write access.

Re: FromDucks - Scrape from I.N.D.U.C.K.S. (V 1.1b) 6 years 9 months ago #11733

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
Ok.. the comic I'm testing the script is Uncle Scrooge #1 (Dell). I am certain that inducks has this.



Earlier the "not found" error showed this as issue US 392. At least now it is showing correctly as US 1



And the familiar error log follows:

Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
Last Edit: 6 years 9 months ago by 600WPMPO.
The administrator has disabled public write access.

Re: FromDucks - Scrape from I.N.D.U.C.K.S. (V 1.1b) 6 years 9 months ago #11737

  • lerichard
  • lerichard's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 2
  • Thank you received: 1
  • Karma: 0
Hi,
I am a contributor to Inducks (and developer as well), and someone pointed me to this page.
First of all, it's great that your program uses Inducks. We've made our data "open" so that anyone may do everything (or almost everything) he wishes with it.
It is highly advised for third-party programs using Inducks data to use the Inducks "ISV" files:
coa.inducks.org/inducks/
They're generated daily and consist of CSV text files. Full information is available here:
bolderbast.inducks.org/xh8.html
The ISV files were exactly made for this purpose, as they contain all Inducks data and their structure is only rarely changed. Note you may not need everything and it may be sufficient to download only a few files.
By contrast, the website coa.inducks.org is generated in part from those files. The COA webpages are changed much more often than the ISV. So downloading these and parsing the HTML, even if it seems easier at first, is probably not a good idea in the long run. I hope you may consider using the ISV files instead.
Apart for that, it's also possible to contact one of us if you need more info.
Note also that if your program uses Inducks data, you should add a notice somewhere that the data is distributed under the conditions of the Inducks licence (I haven't checked, but maybe you are already doing that).
Last Edit: 6 years 9 months ago by lerichard.
The administrator has disabled public write access.
The following user(s) said Thank You: 600WPMPO

Re: FromDucks - Scrape from I.N.D.U.C.K.S. (V 1.1b) 6 years 9 months ago #11738

  • mizio66
  • mizio66's Avatar
  • Offline
  • Platinum Boarder
  • Started reading comics at 4... and still counting!
  • Posts: 451
  • Thank you received: 143
  • Karma: 67
Hi. I did send a mail to the link provided in the pages, I think @googledocs, but at the moment bounces back as waitingr to be delivered.
I obviously want to make the things proper... I knew about the isv files and I use the publications one to fill the main list, but then I thought that in terms of traffic it would have been less demanding going to access the page directly.
If you think it is best to use those, for bandwidth purpose, I will modify the script, no problem.

The script is still in beta, as you see :-( so just let me know.

The credits arein the code, if you agree I can add a (c) in the form, gladly will do!

Thanks for the tips,

M
The administrator has disabled public write access.

Re: FromDucks - Scrape from I.N.D.U.C.K.S. (V 1.1b) 6 years 9 months ago #11739

  • mizio66
  • mizio66's Avatar
  • Offline
  • Platinum Boarder
  • Started reading comics at 4... and still counting!
  • Posts: 451
  • Thank you received: 143
  • Karma: 67
600WPMPO wrote:
Ok.. the comic I'm testing the script is Uncle Scrooge #1 (Dell). I am certain that inducks has this.

I noticed already the different numbering... Will test it as I will be back at the PC...

Again, thanks !

M

P.s. I will remove the huge screen :-) I noticed I doesn't provide that much in for eventually...
The administrator has disabled public write access.

Re: FromDucks - Scrape from I.N.D.U.C.K.S. (V 1.1b) 6 years 9 months ago #11742

  • lerichard
  • lerichard's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 2
  • Thank you received: 1
  • Karma: 0
I'm not exactly sure how your program works but I suspect it's still easier to download a few text files than myriad of web pages. The server also will be less stressed as no dynamic web page needs to be rendered, and I think we have plenty of bandwith.

My email is BTW francois -at- inducks.org

Regards, Francois
The administrator has disabled public write access.

Re: FromDucks - Scrape from I.N.D.U.C.K.S. (V 1.1b) 6 years 9 months ago #11755

  • mizio66
  • mizio66's Avatar
  • Offline
  • Platinum Boarder
  • Started reading comics at 4... and still counting!
  • Posts: 451
  • Thank you received: 143
  • Karma: 67
600WPMPO wrote:
Ok.. the comic I'm testing the script is Uncle Scrooge #1 (Dell). I am certain that inducks has this.
]

600,
this is recognized as Four Colors 386 in CV and Inducks... so unless it is named and numbered like this, it won't be recognized and will fail...

Give it a try...

ciao,

M
The administrator has disabled public write access.

Re: FromDucks - Scrape from I.N.D.U.C.K.S. (V 1.1b) 6 years 8 months ago #12172

  • mizio66
  • mizio66's Avatar
  • Offline
  • Platinum Boarder
  • Started reading comics at 4... and still counting!
  • Posts: 451
  • Thank you received: 143
  • Karma: 67
Latest beta version inserted... v1.5b.

Lots of changes/fixes. Ipersonally rescraped all my DD (Gemstone/Gladstone and Disney) collection... smoothly...

Enjoy.
The administrator has disabled public write access.

Re: FromDucks - Scrape from I.N.D.U.C.K.S. (V 1.5b) 6 years 4 months ago #14877

  • DerMicha
  • DerMicha's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 2
  • Karma: 0
Hi,

what I'm doing wrong? Just installed the 1.5b but when i start i get:

"No module named _abcoll". Then press OK an try again I get "'ScopeStorage' object has no attribute 'FromDucks'". And this repeats with every try...

For testing I downloaded the newest ComicRack version and installed it outside program files dir so I do not expect any windows access problems.

Thanx, Micha
Last Edit: 6 years 4 months ago by DerMicha.
The administrator has disabled public write access.

Re: FromDucks - Scrape from I.N.D.U.C.K.S. (V 1.5b) 6 years 4 months ago #14882

  • mizio66
  • mizio66's Avatar
  • Offline
  • Platinum Boarder
  • Started reading comics at 4... and still counting!
  • Posts: 451
  • Thank you received: 143
  • Karma: 67
Look, I am revising the app to avoid this errors... Be patient some days and I will release a new version... I learned that the best is to try the script on a vanilla pc...
Cheers,

M
The administrator has disabled public write access.
Time to create page: 0.417 seconds

Who's Online

We have 198 guests and 2 members online