Accurate & Comprehensive Database for IR movies & songs

I know it's not an easy task, considering the record keeping standards of the industry (rather industries) plus the magnitude of the output by maestro.

Also, the general pathetic state of affairs in internet for reliability of any data - "தடி எடுத்தவன் எல்லாம் தண்டல்காரன்" business and the information that can be obtained is often inaccurate / incomplete.

One of the reasonably comprehensive records of IR's TFM (though there were many inaccuracies) - thiraipaadal.com - seems to be down and one has no method of knowing if it will ever come back and if so when Even if it comes back, due to the nature of that site (song-play-back-risky-business), one cannot hope to have that database around for ever

Other places like wikipedia, rakkamma etc. all have lots of data but accuracy is very questionable. (For e.g. I was searching for 1990 movies for the vinyl cover thread and rakkamma has "mannan" among them )

Since I don't have any large collection of IR songs anymore, me obviously not qualified to be a source to authenticate such a database. However, I can be the catalyst / motivating force to compile such a database on behalf of our forum - that should satisfy two major criteria :

a) As comprehensive as possible! (Should include movies of all languages / instrumentals / non-filmi stuff etc.)

b) Accurate (Whatever data is presented should be properly verified and authenticated. Corrections should be done periodically to remove any inaccuracies / errors)

With these two main goals, there are other details that will have to be worked out (such as what attributes will be captured, where / how / what format will it be hosted, who will own the database & maintain it etc).

I guess it is an ambitious project but a worthy one IMHO.

We can use this thread as the "collection, discussion, verification, validation, correction etc" place for the project.

Obviously, the support of EACH AND EVERY member of the forum is vital & critical.

Let's first see what each of you think about such an effort..

I think someone in your SPB-IR talked about a comprehensive sheet - wh was it

plum wrote:I think someone in your SPB-IR talked about a comprehensive sheet - wh was it

IIRC, al_gates and he has signed up in our forum as well

We'll request his database and try to use it (along with other open web resources) as the starting ore.

Then do 'pattai theettal' in this thread

I can do my mite. I have mp3 versions of nearly 99% of Raaja's albums including his non-filmy ones except in Telugu. I might not have an idea as to which year the less obscure films released. However, I have the songs and can provide a list of the albums and the songs of each album.

நல்வரவு Drunkenmunk!

Great to have you in this thread!

Thanks to the re-tweets by plum (@nom_d_plum), Nerd (@sagittarian82) & Prem Anandh P (@anandhame) as well as "favouriting" by Spacemen Spiff (@atlasdanced), this thread had ramped up best-possible-no-of-views under the circumstances

I've also got PM-ed by fellow-forumer anbu who has offered his data - collected painstakingly over many years!

I guess we're off to a good start! Being the googling-maniac that I am, the thread will get my best-possible-contribution as well

While "collection of the available data" can start in the background (cannot do that in the thread ), we'll continue to use this thread for interactions and defining the attributes, related queries / suggestions etc.

Let me suggest a few things on the methodology :

#1 Initial data collection tool :

Microsoft Excel I feel is sufficient, to start with. It can later be uploaded to any kind of database on the web. We're talking about a data set not exceeding 1000 albums and 10000 tracks. Even if we assign 100 attributes for each item, Excel can handle it without much ado. What's more, most (I guess all) of the contributors will have access to this tool in one place or another. (And it's easy to use. Also, data can be saved as text / csv and can be ported to any database as desired, later on.) Though some of us may have access to Office 2013, let us share the information in a lower version (2007 or 2010) to make sure most can access it.

#2 Wish-list of Attributes for the albums (whichever is applicable & available ; if not available, we'll leave it blank and fill it if and when we get) :

1. Album / Movie name
   
2. Language
   
3. Year of release of Album
   
4. Year of release of movie
   
5. Movie's director
   
6. Movie's hero / protagonist
   
7. Movie's heroine / protagonist
   
8. Any other key artist associated with the album
   
9. Original / Remake / Dubbing
   
10. Album's overall commercial status (hit / flop)
    
11. Movie's commercial status (hit / flop)
    
12. Special recognitions (SA / NA etc)
    
13. Who updated the record last (audit field)
    
14. When it got updated last (audit field)
    

#3 Wish-list of Attributes for the tracks (whichever is applicable & available ; if not available, we'll leave it blank and fill it if and when we get) :

1. Name of the track (strictly as it is listed in the album)
   
2. Alternate name of the track (if applicable, how it is popularly known as)
   
3. Album it belongs to (and then cross-reference it to the album table / album sheet or have album as well as tracks all in one sheet)
   
4. Male singer(s) - it may be necessary to have fields such as MS1, MS2, MS3, MS4
   
5. Female singer(s) - FS1/2/3/4...I guess 4 will be max
   
6. Male chorus - Y/N
   
7. Female chorus - Y/N
   
8. Lyricist
   
9. Genre
   
10. Ragam
    
11. Thalam
    
12. Any predominant instrument
    
13. Number of saranams
    
14. Prelude Y/N
    
15. Postlude Y/N
    
16. Any special recognitions (award to singer / lyricist etc)
    
17. Hit status (May be Y/N or degrees like patti-thotti-citti / radio only etc)
    
18. Recording tech (Mono / Stereo / DTS etc)
    
19. Last updated by
    
20. Last updated when
    

#4 Initial mode of sharing the "MASTER SPREADSHEET" - share drive on the web (we'll ask V_SJi / plum / groucho for preference)

As usual, I solicit comments / suggestions / observations on the above

fantastic undertaking.. but why not ask someone who is in touch with IR to provide the details? skr?

jaiganesh wrote:fantastic undertaking.. but why not ask someone who is in touch with IR to provide the details? skr?

Good suggestion jaiganesh!

Let's see what skr says sbout this

One great news to share :

Fellow forum member, anbu, has shared his spreadsheet by e-mail to me and IT IS A TREASURE HOUSE!

The speciality is the distinct identification of Audio release year v/s actual movie release year!

He tells me in the mail how it is unacceptable to him to call 'sollaththudikkudhu manasu' as a 1988-er when the audio got released in 1986 and he had been listening to it in TDK cassette months before 1988!

He has such exquisite info as the audio company name & their LABEL NUMBER in many cases

I can now confidently say the project has got an EXCITING KICK START and it won't be long before we have a well-shaped database to show the world!

another good source is film industry chartered accountants.. i think Mohan Raman from the hub (Also a famous actor) is a treasure cove of info when it comes to movies mostly because he is a chartered accountant of producers..

Excellent initiative app ji வாழ்த்துக்கள்! This is sure to have. Let me see what we can about sharing and storing the data.
Many thanks to anbu for sharing the data with us.

nanRi, V_Sji!

At the first glance, IMHO, anbu's spreadsheet will be a great tool to verify & validate the "generally-available-on-the-web" data.

In other words, it contributes greatly to the "Accuracy" goal On the "Comprehensive" part, while it'll be a great help, I can see that it does not have ALL albums / tracks.

So, while we'll be able to use it as a major QA tool, we still need to build ourselves one complete list of albums / tracks from every source available and share it in the place (which you will identify) to start "testing & QA"

Also, IMO, there won't be any "final product" called IRDB

It'll be only a "current-release-IRDB" and will have to have periodic releases / updates / upgrades

Thanks for the great initiative.

Let us not reinvent the wheel. I am maintaining an excel of movies till about 2003. This list was done by Mr. N R Prashanta Rao for his project gem cd stuff.

It can be accessed here. This is a concrete info on Raja's music in all languages. It does not contain his album information


https://docs.google.com/spreadsheet/ccc?key=0AhCVuuFNmJ9ZdGVFRnRxVlY3Zk15OXRPU29SQzlGT3c#gid=0

kameshratnam wrote:
Let us not reinvent the wheel.

Exactly!

Thank you very much for the link!!

Let's also collect all the tracks in a spreadsheet and reference the albums to them and make it available

I'm working with V_Sji to get an initial version of "all-tracks-list" within a reasonable amount of time

This week i went and met Ali khan - the record holder for having all the songs released in tamil cinema.

He is struggling and currently having problems with money. We can try to use his database and also give him some money and make him part of this project. He has all the songs of IR and he charges 250 per cd. A cd may contain 3 movies.

kameshratnam wrote:This week i went and met Ali khan - the record holder for having all the songs released in tamil cinema.

He is struggling and currently having problems with money. We can try to use his database and also give him some money and make him part of this project. He has all the songs of IR and he charges 250 per cd. A cd may contain 3 movies.

i am all up for this..

jaiganesh wrote:

kameshratnam wrote:This week i went and met Ali khan - the record holder for having all the songs released in tamil cinema.

He is struggling and currently having problems with money. We can try to use his database and also give him some money and make him part of this project. He has all the songs of IR and he charges 250 per cd. A cd may contain 3 movies.

i am all up for this..

+1
Methodology / mode / communication can please be co-ordinated by Kameshratnam sir!

I was attempting a similiar one (which raaja fan would not have...the ultimate hobby for a raaja fan)

There are also certain things i had in my excel which probably we can look into like

1. Genre of the song (carnatic, western, blues, jazz, semi classical..), this is obviously tough as raaja usually blends lots of genres (raaja genre). But we can still try it as it would throw lot of surprises
2. Mood of the song (sad, happy, bliss..etc) ..this is something i found very useful from a playlist perspective
3. Song situation (hero-intro, romance, splitup, marriage, pongal, new year etc...)..again this would show the variety of situations and how raaja has handled it differently yet brilliantly each time

All such info will provide excellent view of his work from different angles and allow us to pick and choose whenever we need them

kameshratnam wrote:This week i went and met Ali khan - the record holder for having all the songs released in tamil cinema.

He is struggling and currently having problems with money. We can try to use his database and also give him some money and make him part of this project. He has all the songs of IR and he charges 250 per cd. A cd may contain 3 movies.

Kamesh Sir Does he have a list of movies

Good initiative app. It would be a reference for all future generations.

Great initiative App & co.

Just to clarify im not in close touch with IR
Its just that some info i would have got earlier due to some inside sources

I have seen many lists on the net , but so far have not seen database mentioning the list of songs. (Also there are serious issues with the authentication)
The whole point and the bigger picture makes sense only if you list down the songs too.

My suggestion is to first track down the songs and movie names and then perhaps think of classifying genre/ragam and other stuff coz in many cases there can be no clear cut ragam/genre for a particular song. We can have as many columns as possible but give a higher preference for the base ie the song/movie/yr/singer/language etc.

I feel the basic columns should be
Song No. Movie Number Song Name Movie Name Year Language Dubbed Singer Lyricist etc
1 1 Adi Raakayi Annakilli 1976 Tamil No
2 1 Annakilli Unnai Annakilli 1976 Tamil No
3 1 Annakilli Unnai (Sad) Annakilli 1976 Tamil No
4 1 Machana Paartheengala Annakilli 1976 Tamil No
5 1 Sondam Illa Annakilli 1976 Tamil No

This way we can track down the songs as well as the no. of movies by just doing a filter.
Any better suggestions guys , please list down.

App , From your comments i gather Anbus list sounds pretty good.. You could perhaps share it with us to have a look.
Drunkenmunk , guess you too can share your list here with us.

Thanks
skr

skr,
OK, let me place the link here where I've placed anbu's hardwork:
http://www.mediafire.com/?n5f4bg4rpgmhzmm

Let's keep the discussion on...I agree with you on the "track" based approach

I have uploaded another excel using a macro -vba in excel. When we run it - the macro takes the folder details and the items inside

https://docs.google.com/file/d/0BxCVuuFNmJ9ZTjdycFcyM2tSR0k/edit?usp=sharing

The macro can be taken here

Open the excel file- enable editing and then change the path to the folder where you have the IR songs

http://excelexperts.com/vba-tips-list-files-in-a-folder

app_engine wrote:

jaiganesh wrote:

kameshratnam wrote:This week i went and met Ali khan - the record holder for having all the songs released in tamil cinema.

He is struggling and currently having problems with money. We can try to use his database and also give him some money and make him part of this project. He has all the songs of IR and he charges 250 per cd. A cd may contain 3 movies.

i am all up for this..

+1
Methodology / mode / communication can please be co-ordinated by Kameshratnam sir!

+1 I can Paypal any money. I am yet to get the Kamagni cd from Prashant Rao/Project GEM. When asked for the CD he wanted to know when I paid etc details, which I have lost track of.

kiru wrote:

app_engine wrote:

jaiganesh wrote:

kameshratnam wrote:This week i went and met Ali khan - the record holder for having all the songs released in tamil cinema.

He is struggling and currently having problems with money. We can try to use his database and also give him some money and make him part of this project. He has all the songs of IR and he charges 250 per cd. A cd may contain 3 movies.

i am all up for this..

+1
Methodology / mode / communication can please be co-ordinated by Kameshratnam sir!

+1 I can Paypal any money. I am yet to get the Kamagni cd from Prashant Rao/Project GEM. When asked for the CD he wanted to know when I paid etc details, which I have lost track of.

There are many people who had paid for it and did not receive the cds. I am also one of them. I guess the audio companies screwed it royally and hence they could not deliver.

I will meet Mr. Alikhan and ask him about the details he has and the money which we will require. He will have only the Movie Name Release Year and the singer names i guess - rest we should do.