Welcome, Guest
A place to meet other Developers

TOPIC: Grand Comics Database - API

Re:Grand Comics Database - API 7 years 3 months ago #8605

  • matrixik
  • matrixik's Avatar
  • Offline
  • Junior Boarder
  • Posts: 29
  • Karma: 1
Henry Andrews wrote: source
New (regular) data dumps from the GCD

Hi folks,
The new data dump page is finally ready and open for business! You will find it at:

www.comics.org/download/

The links from the wiki have been updated. In order to ensure that everyone understands the licensing conditions (which remain the same as always) you must now log in to download the dump. We'd also appreciate it if you'd let us know how you plan to use the dump, but those questions are strictly optional. The will help us understand the nature of the customer base for the dumps, and how we might improve the service (beyond what already gets asked on gcd-tech).

The dump should be updated on a more-or-less bi-weekly basis. The updates will happen automatically unless we are having some sort of technical difficulty, which we certainly hope will be quite uncommon.

As always, please either email This email address is being protected from spambots. You need JavaScript enabled to view it. or contact This email address is being protected from spambots. You need JavaScript enabled to view it. if you have any questions or concerns.

thanks,
-henry
---
Henry Andrews
GCD Lead Programmer

Data License and proper credit in derived works
Motion: The GCD should adopt a Creative Commons Attribution (a.k.a. cc-by) license, specifically version 3.0 Unported as documented at creativecommons.org/licenses/by/3.0/ (the "Unported" is a rather inelegant way of saying that it's not specific to any national jurisdiction- the other alternatives, are having various national licenses, or rooting our overall license in one particular jurisdiction).

For our purposes, "Attribution" requires:

* Naming the Grand Comics Database (spelled out) with a link to the main site and the license at least once on each page containing our data. Example:

Data courtesy of the Grand Comics Database under a Creative Commons Attribution license.

where "Grand Comics Database" and "Creative Commons Attribution license" are links. Guidelines and boilerplate for marking usage of cc-by licensed data are available at wiki.creativecommons.org/Marking

* When individual records are listed, a link to the relevant GCD page should be included.
* The Board may waive either or both of the above requirements at its discretion, on a case-by-case basis.
more about license

Current DB schema
Last data dump 2010-06-20.sql have 350 MB.
I am what I am.
The administrator has disabled public write access.

Re:Grand Comics Database - API 7 years 3 months ago #8610

  • spinekill
  • spinekill's Avatar
  • Offline
  • Fresh Boarder
  • Posts: 11
  • Karma: 0
i'd be happy to help in any way i can. I have previous work with databases. I know sql and c#. and can try to learn and pick up anything.
The administrator has disabled public write access.

Re:Grand Comics Database - API 7 years 3 months ago #8611

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
so, cYo.. the team appears ready once again.. whatsay now ?
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
The administrator has disabled public write access.

Re:Grand Comics Database - API 7 years 3 months ago #8612

After months of lurking, this is the thread that convinced me to register.

Long have I been frustrated by the inconsistencies and errors on ComicVine, so this concept is very exciting to me.

I can't help with any of the real hard work (setting up the database and whatnot) but I do pledge to obsessively update metadata.

This might be premature but what are everyone's thoughts on standards? Uh, that was awkwardly phrased...By that I mean, like, I've noticed Titles for certain comics scraped from ComicVine. Some seem to be just the title: Here comes a stranger!

Others use quotation marks or brackets: "Here comes a stranger!" [Here comes a stranger!].

I have to say, the inconsistency drives me batty. I don't know if this is the same for all y'all, but I thought I would bring it up.

There's also the most controversial debate in ecomic-dom: Volumes. I feel if we're going to get this thing up and running, this is something that needs to be tackled early. I personally prefer keeping "Vol. #" out of the Series field. Do we do numbered volumes or year volumes, ala Comic Vine? Year volumes (Vol. 19xx) does do away with some of the debate ("It's Green Lantern volume 4, you jackass!" "No, it's volume 5, you clod!") but I find it looks so...inelegant.

Anyway, I think I've rambled enough. I hope some of this made sense and wasn't...presumptuous?

So excited about this!
The administrator has disabled public write access.

Re:Grand Comics Database - API 7 years 3 months ago #8614

  • quidam
  • quidam's Avatar
  • Offline
  • Platinum Boarder
  • not for nom
  • Posts: 448
  • Thank you received: 23
  • Karma: 30
cYo wrote:
If there are some idle hands here: Why not make a comic database here? And free ourselves from foreign constraints.

I believe this would be the biggest improvement for ComicRack in its current state of excellence. ;) Also, the biggest challenge.

But I think it's worth it. ComicRack really shines when we use metadata to manage our collection. So helping users in retrieving matadata for their comics in more automatic and trouble-free way would be much appreciated. We have ComicVine script and it's a lot of help but it has its disadvantages...

Imagine we have the database set up here. And we can easily connect our library with it. Both ways - to upload new Info we already have filled ourselves for certain series and to download what we're missing. Other community goodies would follow, like sharing comments and reviews for series, etc.

I hope there'll be enough horsepower to make it real. A lot o things would have to be considered... Not only on the developing side, but increase in traffic, server load, all the costs that follow. I know ComicRack is your hobby cYo so you don't want to make money form it, but if the potential costs of this idea are what's concerns you, then consider making it a commercial feature. Small monthly fee for accessing advantages of online database to cover server costs. And some free options for people who contribute a lot. I would go for it and I think I'd be a fair deal.

The least worry is the content itself, as I'm sure there would be a lot of people helping so the database would grow fast (plus the tools to scrap from ComicVine and some manga site, like mangaupdates, and uploading our Libraries to enrich the database).
Last Edit: 7 years 3 months ago by quidam.
The administrator has disabled public write access.

Re:Grand Comics Database - API 7 years 3 months ago #8616

  • 600WPMPO
  • 600WPMPO's Avatar
  • Offline
  • Moderator
  • Posts: 3788
  • Thank you received: 557
  • Karma: 232
quidam wrote:
cYo wrote:
If there are some idle hands here: Why not make a comic database here? And free ourselves from foreign constraints.

I believe this would be the biggest improvement for ComicRack in its current state of excellence. ;) Also, the biggest challenge.

I hope there'll be enough horsepower to make it real. A lot o things would have to be considered... Not only on the developing side, but increase in traffic, server load, all the costs that follow.

I guess quidam has said it all.. B)

Replying on this thread any further is futile till the boss decides something..

Boss ? :huh:
Now Playing: The ComicRack Manual (Online)

See my new comics & gadgets on: Tumblr!
The administrator has disabled public write access.

Re:Grand Comics Database - API 7 years 3 months ago #8618

  • {Oo}
  • {Oo}'s Avatar
  • Offline
  • Platinum Boarder
  • The Chewie is a lie !
  • Posts: 672
  • Thank you received: 41
  • Karma: 15
quidam wrote:
[...]
Imagine we have the database set up here. And we can easily connect our library with it. Both ways - to upload new Info we already have filled ourselves for certain series and to download what we're missing. Other community goodies would follow, like sharing comments and reviews for series, etc.
[...]

Like all "Wiki" type DBs, there'll be a need for constant verification of what's added/modified/removed/etc... If there's none, anyone can put whatever he/she wants wherever he/she wants and the DB will just become useless.

And, as I said, this'll raise the server fees. As Quidam mentioned as well.

And, since I don't know a thing about creating DBs, I'll just imagine that something of that kind would take quite some time. Without even talking about starting to fill it for it to be at least a little useful when it's finally out.

I'm not against the DB idea at all, I'm just playing the Devil's advocate. It's a great idea but with huge constraints.
Working on re-uploading the FR manuals.
Last Edit: 7 years 3 months ago by {Oo}. Reason: forgot something, as usual... =p
The administrator has disabled public write access.

Re:Grand Comics Database - API 7 years 3 months ago #8621

  • deathcroc
  • deathcroc's Avatar
  • Offline
  • Junior Boarder
  • Posts: 22
  • Karma: 0
I just have to say that all this talk of a creating a database that is built to work with ComicRack has my librarian heart all a twitter.

I'm not a programmer, so I can't help that much with that side, but I would very happy to help setup the metadata standards end of things. Creating authority records, determining "best practices", etc.

...


What? OK, so I'm kind of into that type of stuff. That's what us librarians do. :)
The administrator has disabled public write access.

Re:Grand Comics Database - API 7 years 3 months ago #8622

deathcroc wrote:
I just have to say that all this talk of a creating a database that is built to work with ComicRack has my librarian heart all a twitter.

I'm not a programmer, so I can't help that much with that side, but I would very happy to help setup the metadata standards end of things. Creating authority records, determining "best practices", etc.

...


What? OK, so I'm kind of into that type of stuff. That's what us librarians do. :)

I spend more time doing that with my comics than actually reading them. That's what us database analysts do :)
The administrator has disabled public write access.

Re:Grand Comics Database - API 7 years 3 months ago #8640

  • matrixik
  • matrixik's Avatar
  • Offline
  • Junior Boarder
  • Posts: 29
  • Karma: 1
Wow, I didn't expect my topic move in this direction.

To put it simply you want create something similar to:
ComicBase
Collectorz.com Comic Collector
Comic Collector Live!
Realms of Wonder - use comics.com as online database
but better :D
Why?
1. Comics and manga at one place (hurray!)
2. Multi-language

But I see some problems here:
1. What about porn comics and hentai? Mangaupdates show hentai only to registered and adult users, and allow users to filter them.
2. Will you allow adding doujinshi? (Don't know name for comics... Self-published fanzine?)
3. What about different version of same chapter/issue like ashcan editions.

If you would allow adding all that then this db will grow rapidly and soon will be the biggest db at the internet. GCD don't have many different language comics. The Doujinshi & Manga Lexicon (mostly doujinshi) have over 300k objects (more than ComicVine or ComicBookDB have).

Some problems that can appear when designing database are described here: New Fun Schema - GCD

Cheers
I am what I am.
The administrator has disabled public write access.
Time to create page: 0.379 seconds

Who's Online

We have 235 guests and 9 members online