Welcome, Guest
News and Announcements

TOPIC: Comic Vine Scraper 1.0.53-58

Re: Comic Vine Scraper 1.0.53-57 4 years 6 months ago #31329

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
cbanack wrote:
600WPMPO wrote:
I think the scraper should only populate the Released date with the In Store date. It should leave the Published Date alone, till (& if at all) ComicVine decide to open up the Cover Date API for use. Then, the Cover date would be scraped to the Published date (Month & Year, no day). It is my opinion that this would be better instead of getting in incorrect data in the Published date field just for the sake of scraping.
Really? That's a pretty dramatic change, especially for users who don't use the "Catalog" tab. It'll look like they have no publication date being scraped at all...!
Again, with due respect, this is no reason for getting incorrect data just for the sake of filling a field. A simple solution would be to change the default option for showing up the Catalog tab for only fileless books.
cbanack wrote:
The ComicVine guys said they will be providing access to the Cover Date eventually, but they did not say when. So perhaps ComicRack should be modified slighty (cyo?) Maybe replace the Month/Day/Year fields from the details tab with a "Published Date" (or "Cover Date") field similar to the one in the catalog tab?
While it is a reasonable idea to have the drop-down calendar in the published date also, I think (I'm not sure) that this would interfere in grouping by 'Year', and maybe calculating the 'week' field (again, im not at all sure). Other than that, I do not see any practical use for a separate 'month' & 'day' field (as opposed to a drop-down calendar having all 3 in 1). But I can't see as to how this would benefit the current situation. Even with separate day, month & year fields, when ComicVine open up the cover date, we can still scrape, can't we? :unsure:
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
Last Edit: 4 years 6 months ago by 600WPMPO.
The topic has been locked.

Re: Comic Vine Scraper 1.0.53-57 4 years 6 months ago #31330

  • cYo
  • cYo's Avatar
  • Offline
  • Moderator
  • Posts: 3476
  • Thank you received: 675
  • Karma: 181
I will not change any behavior of ComicRack depending if some 3rd party API is complete/implemented/available.

Why is released/in store date on the catalog field? Because it makes not much sense for eComics. And of course, opposite to the published date, it does not have to be universally the same (countries, cities etc.).

Published date is quite important btw, as it is used to sort story lines.

So why not put the released date into the published date if no published date is available. Maybe an option.
The topic has been locked.

Re: Comic Vine Scraper 1.0.53-57 4 years 6 months ago #31331

  • donspace
  • donspace's Avatar
  • Offline
  • Senior Boarder
  • Posts: 67
  • Thank you received: 17
  • Karma: 7
cYo wrote:
So why not put the released date into the published date if no published date is available. Maybe an option.

This makes a lot of sense on a couple of different levels. Not only is it "maybe an option" for populate "published date" with "release date", it could physically be an actual option in 'Scraper. There are several behavior options included in the script by cbanack; make this an option as well. If you're a purist like 600, you would not use the option. If you must have "published" date--as a sort option for example--then you can opt in by checking a box to "populate publish date with release date when/if publish date unavailable".

D
Java devil, you are now my bitch.
The topic has been locked.

Re: Comic Vine Scraper 1.0.53-57 4 years 6 months ago #31332

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
cYo wrote:
So why not put the released date into the published date if no published date is available. Maybe an option.
Why not put the penciller's name into the letterer's name if no letterer is available...:silly:
donspace wrote:
...it could physically be an actual option in 'Scraper. There are several behavior options included in the script by cbanack; make this an option as well....
This I agree with. :)
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
Last Edit: 4 years 6 months ago by 600WPMPO.
The topic has been locked.

Re: Comic Vine Scraper 1.0.53-57 4 years 6 months ago #31333

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
600WPMPO wrote:
Again, with due respect, this is no reason for getting incorrect data just for the sake of filling a field.
Well, the scraper has been populating day/month/year since it was first created, and I'm pretty sure people (including myself) use those fields for all kinds of smart lists/sorting/etc. If the data in these fields suddenly disappears without any kind of replacement, I have a feeling I'll be be buried under a mountain of complaints. Moreover, it's not like anything is changing here; the scraper has already been putting ComicVine's "published_date" into the day/month/year fields for years.

However, once ComicVine's API starts provides "cover_date" and "instore_date", it would make perfect sense to scrape them the way you have already described.
A simple solution would be to change the default option for showing up the Catalog tab for only fileless books.
Yes, this would help. Another option would be to move the Released Date out of the Catalog tab and into the Details tab, since it seems like it applies to all comic books anyway. (EDIT: ahh, I see. Doesn't really work for eComics.)
But I can't see as to how this would benefit the current situation. Even with separate day, month & year fields, when ComicVine open up the cover date, we can still scrape, can't we? :unsure:
Yes, no matter how cYo decides to configure the date fields, we can still scrape data into them.
Last Edit: 4 years 6 months ago by cbanack.
The topic has been locked.

Re: Comic Vine Scraper 1.0.53-57 4 years 6 months ago #31334

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
donspace wrote:
cYo wrote:
So why not put the released date into the published date if no published date is available. Maybe an option.

This makes a lot of sense on a couple of different levels. Not only is it "maybe an option" for populate "published date" with "release date", it could physically be an actual option in 'Scraper. There are several behavior options included in the script by cbanack; make this an option as well. If you're a purist like 600, you would not use the option. If you must have "published" date--as a sort option for example--then you can opt in by checking a box to "populate publish date with release date when/if publish date unavailable".

D
Yes, I think this is what I will do. However, I still need a bit of a hint from cYo on how to populate the "Release Date" field. There is no book.ReleaseDate property, and book.ReleaseTime doesn't seem to work.

EDIT: I'll just remove the "day", "month", and "year" checkboxes from the settings dialog, and replace them with "Published Date" and "Released Date". And until ComicVine starts giving me access to a proper Published Date (i.e. Cover Date) I will scrape their "In Store Date" into both places.
Last Edit: 4 years 6 months ago by cbanack.
The topic has been locked.

Re: Comic Vine Scraper 1.0.53-57 4 years 6 months ago #31339

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
cbanack wrote:
Well, the scraper has been populating day/month/year since it was first created....t's not like anything is changing here; the scraper has already been putting ComicVine's "published_date" into the day/month/year fields for years.
The scraper has been doing the correct thing. ComicVine's "published_date" corresponds to the day/month/year fields we used to have for years. Then the released date field came along & we had nothing to scrape in there, as ComicVine did not have any scrapable info for that field. And then, the redesign came along, and now we've got a scrapable 'released date' field, and (for now) an empty "published_date" (the day/month/year fields).

We just have to wait. If users complain that they have got an empty published date, they have to be told to wait till ComicVine lets us fill it up (just like we are telling them for creator fields). If they HAVE to fill it up before that, they can use a script like 'copy-move fields' to copy the released date to the published date.

Suppose, we get the in-store date into both the fields, and later on when we get the cover date scrapable, we'll have to rescrape (unless we are happy with an incorrectly filled field, as opposed to an empty field). So, whether we leave it empty for now or not, In any case, we'll have to rescrape.

I totally fail to understand how can one agree to get a 'published date' of, say, 'March 2013' when the cover/indicia clearly says 'April 2013'?

Please don't misunderstand me. Even if you eventually go ahead with filling up both fields with the same date, I can easily empty/modify the published date (with or without an option in the scraper's preferences). But, your doing so would not be authentic/accurate. It is a matter of principle.
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
Last Edit: 4 years 6 months ago by 600WPMPO.
The topic has been locked.

Re: Comic Vine Scraper 1.0.53-57 4 years 6 months ago #31340

  • cYo
  • cYo's Avatar
  • Offline
  • Moderator
  • Posts: 3476
  • Thank you received: 675
  • Karma: 181
I complain right now if the published date is empty...

@cbanack:

Comic Books have three full blown dates (Published is a special case):

Book.AddedTime -> When has the book been added to the libary
Book.ReleasedTime -> In Store time (but only date part is saved)
Book.OpenedTime -> When has the book the last time been opened

I agree that in hindsight Book.ReleaseTime may have been better, but the current name is now out in the wild :)
Last Edit: 4 years 6 months ago by cYo.
The topic has been locked.
The following user(s) said Thank You: dionysuswild

Re: Comic Vine Scraper 1.0.53-57 4 years 6 months ago #31341

  • forkicks
  • forkicks's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 869
  • Thank you received: 108
  • Karma: 37
600WPMPO wrote:
It is a matter of principle.

Not so, because it also breaks the scraper's expected behaviour. If it has been running as it is currently since ever, then it shouldn't start doing something different from one version to another, at least without the user explicitly telling it to.

I wouldn't oppose a user setting for this, but i would be very surprised if all my lists that rely on dates suddenly broke because this caught me blindsided (by "me", i'm obviously meaning Joe User).

fK
The topic has been locked.

Re: Comic Vine Scraper 1.0.53-57 4 years 6 months ago #31346

  • cbanack
  • cbanack's Avatar
  • Offline
  • Platinum Boarder
  • Posts: 1318
  • Thank you received: 503
  • Karma: 181
cYo wrote:
Book.ReleasedTime -> In Store time (but only date part is saved)
Do I need to do something special to change this value? I tried this last night and nothing changed?
book.ReleasedTime = DateTime(1999,1,1)
forKicks wrote:
Not so, because it also breaks the scraper's expected behaviour. If it has been running as it is currently since ever, then it shouldn't start doing something different from one version to another, at least without the user explicitly telling it to.

I wouldn't oppose a user setting for this, but i would be very surprised if all my lists that rely on dates suddenly broke because this caught me blindsided (by "me", i'm obviously meaning Joe User).
Yeah, I see what 600 is saying (the two dates are not the same thing, period) but software engineering is an art of compromise; I can't make a choice based on principle when the practical reality of the matter would be so frustrating.

Fortunately, we have a good compromise that should make everyone happy: I'll remove the "Day", "Month" and "Year" checkboxes from the scraper's settings, and replace them with "Published Date" and "Released Date" checkboxes. For now, both date fields will be filled with ComicVine's "In Store Date", but when the "Cover Date" becomes available, I'll change "Published Date" to use that instead. All good, right?!? :laugh:
Last Edit: 4 years 6 months ago by cbanack.
The topic has been locked.
Time to create page: 0.293 seconds

Who's Online

We have 168 guests and 5 members online