XMLTV Listings importer for non-US users

[mrphil]

This program takes the XMLTV formated listings file, and adds that data to the PVS EPG. The utility does not scrape listings itself; it only converts existing files. There are reasons of legality which prevent the actual scraping of listings within this utility, it is therefore the users responsibility to supply data. I recommend the use of the XMLTV utility. XMLTV can be downloaded here (http://sourceforge.net/project/showfiles.php?group_id=39046).

If you want help setting up, please read Percy Bell's excellent setup guide here (http://www.snapstream.com/Community/Articles/xmltv/xmltv.asp)

This utility is now hosted on a seperate site. See http://pvsxmlimport.sourceforge.net/. You should go there for the latest builds and source. Site kindly setup by Ctebo.

Admin note: This product is NOT supported by Snapstream staff.

[Tuck]

This is an addition to the previous article, and is taken from another post. To any users reading this who discover a discrepancy, please send me a Private Message to let me know if I need to modify this information:


The XMLTV grabber supports:
Canada, UK, Germany, Austria, Sweden, Norway, Finland, New Zealand, Spain, Italy and the Netherlands.

Appparently comming soon are Belgium, Denmark, France, and Hungary.

[Sludge321]

Thanks a lot for this tool. It works well - excellent!!!

Cheers again.

[Scott Anderson]

Mr Phil,

Thanks for your work on the XMLTV importer. I was happy as a clam using your software, but now for some reason it is not working for me. During the conversion of my .xml file, Importer just hangs and never completes.

I was wondering if you have had any experience with this problem or can give me some insight.

Thanks again.

[mrphil]

How big is the file you are using? lots of channels * lots of data = long time processing. I have seen it take up to an hour and a half to process data. Also, you could replace your database in the PVS directory with a clean one, but you will need to re-enter your channel mappings.

[Scott Anderson]

Actually, I tried replacing with a clean database file and only using one days worth of data x 50 channels. Either way, same problem. Within 10 seconds of processing the guide data the application hangs.

[i0900j2]

Hi,

in the importer-zip was no "ss_pvs_db.mdb" and I cannot find that file on my harddrive as well (non-us user). Without this file it seems to be unimpossible to succesfully run through the starting-process of the importer.

Any suggestions - or could some just mail me a ss_pvs_db.mdb?

Thanks

Till

[jkoon]

If you are using windows 2000 or XP then it ‘should’ be in this folder:

C:\documents and settings\all users\application data\SnapStream\Personal Video Station\

I’m not sure where it is at off the top of my head for windows 98.

[i0900j2]

Thanks for your help, but I've been searching the whole hard drive, this file is just not there ...

Till

[mrphil]

On Windows 98 it is in c:\windows\profiles\all users\application data\snapstream\personal video station\

If its not there you need to reinstall PVS.

The file is no longer included in the zip, thats why its not in there.

[i0900j2]

Thanks for your reply, I actually reinstalles pvs today, but the file is nowhere, so I still need it. By the way, I' m using W2K.

Maybe someone could just post it ...

Till

[James]

c:\documents and settings\all users\application data\snapstream\personal video station should be where it is located on win2k/XP..

[i0900j2]

You are not getting the point: I found this folder, but there is no such file in there. Nor anywhere else on my harddrive, since I have been searching the whole drive using the search function. I think it's really not there ...

[Scott Anderson]

Are you sure you read Mr Phil's last post entirely? You need to reinstall PVS if this file is missing.

[mrphil]

Originally posted by i0900j2
You are not getting the point: I found this folder, but there is no such file in there. Nor anywhere else on my harddrive, since I have been searching the whole drive using the search function. I think it's really not there ...

"Application Data" is a hidden Windows system directory, so any files in it also inherit the hidden attribute. By default the Windows Search function will not search hidden files and folders, so Im not surprised you can find it using that method.

Yopu can confirm the file is there by going to Control Panel, open ODBC Data Sources, click System DSN, click SS_PERSONAL_VIDEOSTATION (or something similar) click configure, and check the path there.

[i0900j2]

Ok, thank you that helped. Those folders are named quite differently as I'm running a german version of windows. Now it works fine. Thanks again.

[Mr.X]

For some reason, the chanlineup.txt file has 144 stations, many of them repeats, and they're not in order. (XMLTV only downloads 60 channels)

What am I doing wrong?

-Mr.X

[mrphil]

Shutdown PVS

Delete ss_pvs_db.mdb in c:\documents and settings\all users\application data\snapstream\personal video station

Copy ss_pvs_db.mdb.clean and rename to copy to the original filename ss_pvs_db.mdb

Run importer and enter channels as you are prompted.

[lchiu7]

I have followed all the steps to grab the xml TV listings using the appropriate grabber, created the xmltv.dtd and then ran the import program. That works without a hitch and the snapstream ss_pvs_db.mdb appears to be the right size. In snapstream.net settings I have disabled automatic updates.

However when I select Program Guide it always want to logon onto snapstream.net and not display the program guide data I have loaded. What else do I have to do?

Thanks

Larry

[mrphil]

As stated in the documentation, this importer is for the local EPG you see in the full screen interface ONLY. The programme guide in the web admin is just a link to ss.net. You must provide your own web based epg if you want to use one from the web admin screens. See this (http://discuss2.snapstream.com/vb/showthread.php?s=&threadid=7805&perpage=15&pagenumber=2) thread for PeterOA's web based epg viewer

[lchiu7]

Yes, thanks. I finally worked it out after playing around with the settings and interface

Thanks

Larry

[htsource]

I was able to run the importer for a while and now it stopped working. I no longer get EPG data into Snapstream and the current size of the database is about 44MB. I tried using MS Access to compact the database but it requires the password.

I can try using a clean database again but I'm trying to find a way to avoid re-entering all channel information if I don't have to.

Thanks,

Simon

[mrphil]

Simon, I have sent you a PM.

[htsource]

mrphil,

If you don't mind, could you please resend the PM? When I clicked on PM icon, it told me I have the function off. I just turned it back on so I should be able to receive messages.

Thanks a lot for your help,

Simon

[htsource]

This is strange. I just found out even though the TV Gudie shows everything "not available", yet if I watch live TV, whenever I up/down a channel, it shows the information about the show.

I have 2 Video Inputs set up (one for satellite and other for cable TV). For some reason, satellite line up comes up all the time in EPG. How can I change it so the cable EPG shows up instead?

It used to work just fine though. Thanks for your response,

Simon

[mrphil]

Eh? I really don't see how that is possible, unless your satellite and cable channel numbers do not duplicate each other, in which case I could understand that you had, for example, channel 1 -99 for cable showing only for one lineup, and only numbers beyond 100 for satellite showing for the satellite lineup. The PVS EPG does not actually allow for more than one lineup, it can only work if your various lineups are all using unique channel numbers. Whew! Hope all that makes some sort of sense!

[Per]

I've just imported some new data. When I check the recording manager and upcoming recordings, not all shows are marked to be recorded. How do I troubleshoot this problem?

[mrphil]

What sort of recordings are they that are missing:

- One off, or record first run, or record all?

What is the first run/aired date?

Also, how you tried doing a restart of PVS?

[Per]

It's "record all" and the first run date is this thursday.

And yes, PVS is restared.

btw, what's best? PVS running or not during import?

[mrphil]

What do you see in the FSUI EPG grid marked in those programmes? Is it Just an "A", or an "A with a line through it"?

Also, as far as importing goes, it makes no difference if PVS is running or not.

[htsource]

I have 2 lineups for sat and cable. Cable is from channel 2 - 68 and sat is from 200 - 970.

For some reason, EPG shows channels from 200 - 970 only. Nothing from 2 - 68. For each video input, I specifically select the line up for each so if I want to watch Live TV, it changes channel depending on what I'm watching (i.e. shows correct listing for cable).

The strange thing is I get "not available" for the EPG because XMLTV grabs channels from 2 - 68 only while it's showing 200 - 970 channel listing. When I watch live TV through sat, I get nothing for the program description. However, when I watch cable TV live, it shows the description of the program.

This proves the plugin is getting data from XMLTV, however, how do I change the EPG guide so it only shows listing for cable TV and not sat TV.

I hope this clear things a bit, thanks again for your help.

Simon

[Per]

I see an "A with a line through it"

[mrphil]

Try importing the same data again. I had this problem a week or two ago, and re-importing fixed it, although I cant think why. I will look into this further next time I can reporduce it.

[srigby]

ImportXMLTV is a great app, and I know have a source for XMLTV listings.

However, when it grabs a number of days worth of data, it writes it out to individual files, ie: 20030728.xml, 20030729.xml, etc...

Is there any chance of ImportXMLTV beind modified to support this format?

I've already asked the XMLTV source to write multiple days out to one file which was his original intention - but for some reason he was unable to do it.

If you don't ask, you don't get ;-)

[mrphil]

Post a file so I can look inside it, if the files are using the xmltv standard, they should already work without modification.

[srigby]

Oh, the files work fine. Guess I shouldn't have used the word format.

ImportXMLTV is looking for a single filename (ie: listings.xml) which contains multiple days worth of guide data.

However, the XMLTV grabber app will go and get five days worth of data and write five seperate files! How am I supposed to write a batch file to run the whole thing automatically?

I thought about adding the five files together, but couldn't think of a syntax that would work reliably.

Thanks for showing some interest anyway....

[mrphil]

What if I change the code so it also allows you specify just a directory, and it will import all files it finds in that directory? W=Is this what your looking for?

Im thinking also would it be a good idea to delete each file after it has been processed, so you don't import it again next time things run.

[srigby]

Brilliant!

Either my batch routine can delete the already processed .xml files after ImportXMLTV has finished - or you might want to build that into the code, ie: renaming processed .xml files to .old

Thanks for the willingness to help, and for doing ImportXMLTV in the first place!

[catdna]

Hi Guys,

Just a quick note before I get down to my point - I'm considering building a PVR myself, and the snapstream software looks a likely candidate for the interface I'd like.

I've not had a chance to download it yet, but I've been reading the site and the ImportXMLTV program interested me (being based in the UK myself).

I downloaded the program (and code) so I could see what's going on.

I can now see several reasons why some people report the program to be running slowly.

As a professional VB programmer myself (among other languages), I've spotted several 'gotcha's' and areas of improvement.

I'd be quite willing to rewrite the program myself (if mrphil) doesn't mind - or I can give you the pointers here (or just mail me).

I'm going to download the software (snapstream) this evening anyway and have a play (no PVR card at the moment, but at least I can (hopefully) play with the GUI).

The one thing that does intrigue me however, is that mrphil's code includes the password for the epg database - I presume that this is public knowledge ? or is it hacked ;-)

Anyway, let me know if you want a hand with the ImportXMLTV program (or if you'd like me to come up with a newer version) - like I said, I'm quite happy to help out.

Cheers

Chris Eastwood

[mrphil]

Originally posted by catdna

I can now see several reasons why some people report the program to be running slowly.

As a professional VB programmer myself (among other languages), I've spotted several 'gotcha's' and areas of improvement.


Yeah, I know they are there. I simply don't have the time to sort this out. Much as I would love to sit down and clean up the code, Im flat out doing an Active Directory migration at the moment. Thats why I posted the source, so if someone else needed to add something or fix a bug, they don't have to wait for me.

The whole project was effectively a reverse-engineer of the way PVS uses its epg with some pointers from James@SS, so it can definately be done better, but at the time people wanted it yesterday, so much of the alpha code made it into the release depite it being crap.

I'd be quite willing to rewrite the program myself (if mrphil) doesn't mind - or I can give you the pointers here (or just mail me).

Please do, if you think you can make imporvements. Just make sure you send it to me so I can update the first post of this thread, and let the whole community benefit. Take a good hard look at the routines which generate the various codes used in the database, they must be perfect or some features of the EPG fail. They are not random!

The one thing that does intrigue me however, is that mrphil's code includes the password for the epg database - I presume that this is public knowledge ? or is it hacked ;-)

Its hacked. I should probably take that out.

[catdna]

Please do, if you think you can make imporvements. Just make sure you send it to me so I can update the first post of this thread, and let the whole community benefit. Take a good hard look at the routines which generate the various codes used in the database, they must be perfect or some features of the EPG fail. They are not random!


I figured as much - I can see what's going on in the EPG episode codes and think I understand it - probably best to wait until I've actually installed snapstream before I (re)write the code though :)



Its hacked. I should probably take that out. [/B]

I thought as much :)

Will the snapstream peeps get 'annoyed' at the use of this code ? (I guess not as they link to it from their site). I can encrypt the password in the source code (although it wouldn't take a genius to work out the password).

Might be worth moving it out to a separate file for when / if the powers that be change the password.

I'll mail you the finished code when it's done - won't be for a while as I'm busy too, but should have something for next week (and I need time to play with snapstream :))

Cheers

Chris

[olaf]

Hi, I used to work very well with the importer, but now that XMLTV has updated their XML it no longer works. Does anybody have a newer version that does work? Love to have it work again.

Olaf

[mrphil]

They have? I will look into that, in the mean time could you not just stick with the older version of xmltv?

[olaf]

It look like it to me. It now keeps asking me where each channel needs to go, it will ask me this for all the URL's in the XML. I tried with the old one but will do the same. Is this a different format or is something else wrong on my PC? This comes from the NL both _nl and _nl_wolf

The old xmltv-0.5.11-win32 seems to give the same format now.

It now has a channel option before it like:

<channel id="6.tvgids.nl">
<display-name>KETNET/Canvas</display-name>
</channel>
<channel id="89.tvgids.nl">
<display-name>Nickelodeon</display-name>
</channel>
<programme start="20030804064300 +0300" stop="20030804070000 +0300" channel="1.tvgids.nl">
<title lang="nl">Nederland in beweging</title>
<url>http://www.tvgids.nl/detail.php?pid=695466&amp;tab=1</url>
</programme>
<programme start="20030804070000 +0300" stop="20030804071500 +0300" channel="1.tvgids.nl">
<title lang="nl">NOS-Journaal</title>
<url>http://www.tvgids.nl/detail.php?pid=695467&amp;tab=1</url>
</programme>

[mrphil]

attach an actual file. Theres nothing wrong with what you have posted, that is the normal XMLTV format. It looks like there is some issue on your system, possibly an outdated version of the MSXML parser, or perhaps a corrupted database. Did you try deleting the existing SS_PVS_DB.MDB file and replacing it with SS_PVS_DB.MDB.CLEAN?

[Gman]

Originally posted by mrphil
Try importing the same data again. I had this problem a week or two ago, and re-importing fixed it, although I cant think why. I will look into this further next time I can reporduce it.

Just a note that I regularly get this problem. Reimporting sometimes fixes it, sometimes not. For example I had the problem on four shows after todays import. I ran it again and only had problems on two.

Interestingly, you usually cannot force these "crossed out" shows to record from the "Show upcoming episodes" screen, but you can set a manual recording for the same time to get around it.

Hopefully you can find the problem at some stage, mrphil. Regardless, your wonderful little program is much appreciated!

[mrphil]

Per has been sending me updates on this as he is experiencing this also. A work around looks to be to delete the file ExceptionsRecordingCollection.xml from your machine. if you are using a batch file to run xmltv and then the importer, I would suggest adding a final line of

delete c:\documents and settings\all users\snapstream\personal video station\ExceptionsRecordingCollection.xml

which will remove the file that is storing the previously recorded shows from your system and allow recordings to be made.

[Gman]

You didn't miss a step did you? I just renamed that file, but it didn't make any difference. I even stopped and started PVS - it's still unhappy about the same shows. Looking in that file, it has most of the problem shows in there, but not all.

I'm running another import now, so we'll see how we go!

<edit>It seems that two shows still have glitches. *sigh* Your method did knock off some of the trouble shows, so thanks for that!</edit>

[mrphil]

Hmm, try this one instead:

SlaveStoreCollection.xml

But make a backup first! It might remove all your recordings, I cant try this myself at the moment.

[Gman]

Hmmm. I made a backup of that one and deleted the original, but it hasn't made any difference.

Interestingly, SS hasn't recreated it, although the video library is quite happy (that's the file with the list of what in your library, right?) even though the XML hasn't been recreated.

I didn't stop and restart SS completely this time, as it's recording a show.

[mrphil]

Hmm, dont know then. I will try and recreate this on my machine later and look into it further.

[olaf]

I still have a problem importing. When it asks me to asign a channel number to the channels something goes wrong. I tried the clean database.

First it will ask for for the normall channels, this goes well. Their are also a lot of new channels (when they where not their the import worked very well) I don't have most of the extra channels so I asign a number to then that I don't use. When it has asked for for all the channels it will start asking me for all the enters of all the channels that I don't have. It will then show the URL of the channel and ask for a number. I have done this once for all (was a very big job) and then it did import all into the database.

SS still didn't show the EPG. I looked in the database (got the pass from your code) and all the EPG data was their. The only thing I could see missing was that their was no record from the last import and if it was sucsesfull.

I am attaching the XML file, not sure if the error is their. I have not updated my PC and it used to work fine, till the extra channels where added to xmltv.

Hope somebody can help me, need to have my EPG. :-)

Olaf

[mrphil]

Olaf, I tried the file you posted, and it works fine. It imported correctly and works fine. From the information you gave, everything is going OK. I would suggest you check the following, in order until things work:

1) In the Web Admin of PVS, make sure that Automatic Updates under Snapstream.Net settings, is set to Disable. This is critical, it will not work otherwise.

2) Check your clock date and time are correct.

3) Delete any callsigns you have entered into your channel lineup in the Web Admin.

Also, you must be running a full version of PVS 3.0. Trial versions are not supported.

[olaf]

I wanted to thank you for your great help. I know you get many questions and it seems like you put in a lot of time. I think it is great their are people like you helping others this way.

I guess it was because I have put in some call signs, when I took those out it fixed it.

Again thank you for your time.

Olaf

[wildbeast]

Anyone having trouble using XMLTV lately? I've been getting www site may be down error since yesterday.

It says "zap2it failed to give us a page: 500:Internal Server Error"

[mdorion]

Originally posted by wildbeast
Anyone having trouble using XMLTV lately? I've been getting www site may be down error since yesterday.

It says "zap2it failed to give us a page: 500:Internal Server Error"

It seems zap2it.com changed some internal element of their site (URLs, cookies, dunno!) in a way which breaks xmltv for NA (North America). See bug reports on the xmltv page at Sourceforge : http://sourceforge.net/tracker/index.php?func=detail&aid=786963&group_id=39046&atid=424135

:mad:

One more reason for the fine Snapstream folks to really support Canada ASAP.

Mario

[PeterOA]

Looks like someone has come up with a patch for the Win32 executable. Follow the thread above and there is a URL for the file - I believe it is alpha-exe.xmltv.org. I have run it and it appears to work fine.

P.

[mdorion]

Originally posted by PeterOA
Looks like someone has come up with a patch for the Win32 executable. Follow the thread above and there is a URL for the file - I believe it is alpha-exe.xmltv.org. I have run it and it appears to work fine.

P.

Quite correct, it does work :D

[Eagle0]

Ok. I'm having troubles with one step here..

"Using the grabber for your country, at the command prompt type:"

"C:\XMLTV> xmltv tv_grab_uk --configure"

I'm trying to use the grabber for the u.s.. I tried and get: tv_grab_us_gist is not a valid command. Valid commands are:....

Can someone tell be what I'm missing here.. thanks..

[mrphil]

Originally posted by Eagle0

I'm trying to use the grabber for the u.s.. Why? Snapstream provide a proper listings service in the US!

[Eagle0]

Ok.. Actually I'm going to want to bounce back and forth from us and canada. What is best way for this? Stick with a normal setup(snapstream). Just feeling out options and soaking up (trying to) as much info possible here..

And to add to this. I'm trying ti understand the xmltv for the near future, when I tackle the linux side..

Thanks

[mdorion]

Originally posted by Eagle0
Ok. I'm having troubles with one step here..

"Using the grabber for your country, at the command prompt type:"

"C:\XMLTV> xmltv tv_grab_uk --configure"

I'm trying to use the grabber for the u.s.. I tried and get: tv_grab_us_gist is not a valid command. Valid commands are:....

Can someone tell be what I'm missing here.. thanks..

tv_grab_na (North America) is for both US and Canada

[peter]

I was just wondering whether other people were getting callsigns (channel names) in the EPG.

Obviously we can't add callsigns into the web admin, as that breaks the xmltv import process.

Is the xmltv import app supposed to populate the EPG with channel names? I assume snapstream.net does this. Its annoying having to refer to channels via a number rather than a name.

Thanks
Peter

[svickory]

Unless I am doing something from XMLTV doesnt have DirectV listing for PPV either? is this correct?

[mrphil]

Originally posted by peter
I was just wondering whether other people were getting callsigns (channel names) in the EPG.

Obviously we can't add callsigns into the web admin, as that breaks the xmltv import process.

Is the xmltv import app supposed to populate the EPG with channel names? I assume snapstream.net does this. Its annoying having to refer to channels via a number rather than a name.

Thanks
Peter

Actually, you can. The first run of the importer lets you enter the channel numbers - at the same time it lets you change the channel callsigns to whatever you want ( as long as it is unique). These details are written to a file called chanlineup.txt or something like that. You take the callsign for each channel and enter it into the callsign field for the appropriate channel in the web admin, and you get on screen callsigns. Thi must be identical, letter for letter. Copy and paste works best. Any deviation and it wont work. This isnt really talked about much as its fairly labourious and is a big sticking point for a lot of new users.

If you want to CHANGE the callsigns, you must start again with a new DB and re-enter all your channels, OR open the DB in access and edit the callsign field.

[mrphil]

Originally posted by Eagle0
Ok.. Actually I'm going to want to bounce back and forth from us and canada. What is best way for this? Stick with a normal setup(snapstream). Just feeling out options and soaking up (trying to) as much info possible here..

And to add to this. I'm trying ti understand the xmltv for the near future, when I tackle the linux side..

Thanks

Ah, I see. You wont be able to do that Im afraid. Using XMLTV and Snapstream data in the same database is a killer for PVS.

[svickory]

Do you know if XMLTV has the PPV names for Directv? I am in north america, also can you check your email, and pm I was wanting your help writing a program. and paying you for it :-)\

[mrphil]

Originally posted by svickory
Do you know if XMLTV has the PPV names for Directv? I am in north america, also can you check your email, and pm I was wanting your help writing a program. and paying you for it :-)\

No Idea. I cannot provide any information on the XMLTV application. I just wrote an importer to use AMLTV formatted files. I would suggest downloading xmltv and reading the associated documentation. Also, if you went this route, you would not be able to use Snapstream.net listings at all. You can have one or the other, not both. Personally, if I had a choice Id stick with Snapstream listings and be without the PPV channel listings.

[mrphil]

Ok, so here is a possible workaround for the EPG DATA is low message. Perhaps someone can try this out and get back with the results.

Open up the PVS epg database, go to the update table, and change the END field so the date is 3 - 4 weeks after the START date. You will then need to restart PVS for changes to take effect. Note the format the date is in, its standard XMLTV style, so make sure you conform to this or it wont work.

[Shaddow]

First sorry for my english.
Im spanish and im trying to configure.... but without results.

XMLTV create a file of 130K:

.................................................. ...
<?xml version="1.0" encoding="ISO-8859-1" ?>
<!DOCTYPE tv (View Source for full doctype...)>
- <tv source-info-url="http://www.elpais.es/parrillatv/portada.html" source-data-url="http://www.elpais.es/parrillatv/resultados.html" generator-info-name="XMLTV" generator-info-url="http://membled.com/work/apps/xmltv/">
- <channel id="01.elpais.es">
<display-name>TVE1</display-name>
</channel>
- <channel id="02.elpais.es">
<display-name>La 2 de TVE</display-name>
</channel>
- <channel id="03.elpais.es">
<display-name>Antena 3</display-name>
</channel>
- <channel id="04.elpais.es">
<display-name>Tele 5</display-name>
</channel>
- <channel id="05.elpais.es">
<display-name>Canal Plus</display-name>
</channel>
- <channel id="06.elpais.es">
<display-name>Telemadrid</display-name>
</channel>
- <programme start="20030828060000 +0200" channel="01.elpais.es" clumpidx="0/1">
<title lang="es">Canal 24 horas</title>
<desc lang="es">Informativo.</desc>
<category lang="es">INFORMATIVO</category>
</programme>
.................................................. ...

when ran xml import the first time, found 6 chanels but 0 programmes.
In the program guide tv of PVS all channels information shows unavailable.

What is the problem?

PD: i see that the xml of xmltv dont have time stop..... would be it the problem?

[mrphil]

Sorry, but it wont import programmes unless they have both a START and END time. Perhaps there is an alternative source for your listings?

[Shaddow]

For sources that dont add the stop time of the programm, i made a program that add it. I think that could be interesting for someone, file is attached.

Now, i can view tv guide :)

[Christoph_vW]

I replaced the ComDlg32.ocx by a selfmade UserControl which wrapps the ComDlg32.dll API Functions.

No Setup required, smaller filesize.

Here is the new source...

www.ApiViewer.de/downloadstore/importsourcenew.zip

[earthling0001]

Originally posted by mrphil
Ok, so here is a possible workaround for the EPG DATA is low message. Perhaps someone can try this out and get back with the results.

Open up the PVS epg database, go to the update table, and change the END field so the date is 3 - 4 weeks after the START date. You will then need to restart PVS for changes to take effect. Note the format the date is in, its standard XMLTV style, so make sure you conform to this or it wont work.

Hi, new here, looks like fun! :p A few Qs:

1) Am I right in think I will also get the XMLTV error above that there is low EPG data, if I am running the Lite trial version of PVS? That is what I have, and I've succesfully downloaded and run the import, but nothing appears in the Program Guide.

2) Can someone PM me the password for the DB please.

3) Is there any 'Wishlists' feature where I can enter programmes I want it to record when they appear in the guide that aren't currently listed, such as on my (probably soon to be ex-)TiVo?

4) Why do I only get 3-4 days of listings downloaded from Ananova? TiVo gives me 21 days avg. This is a major downside of PVS. Any other listings sources I can try with XMLTV?

5) Is there a way, using a dual-head video card, to use the computer to generate output to 2 screens, screen 1 showing the computer Windows desktop so user 1 can use that, whilst screen 2 shows the PVS output so user 2 can watch TV, all from one PC simultaneously? Otherwise it makes sense to keep my TiVo if I have to get off the computer when someone wants to watch TV, even though we have 2 screens.

Thanks!

[TimFrancis]

Originally posted by Shaddow
PS: i see that the xml of xmltv dont have time stop..... would be it the problem?

I had a similar problem - and found that the XMLTv utility 'tvsort' will actually add stop times for you (based on the start time of the next program on the same channel).

My 'import' process now consists of:
call xmltv tv_grab_na --output raw.xml
call xmltv tv_sort --output na.xml raw.xml
call ImportXMLTv

And it seems to work great. Hope this helps.

[TimFrancis]

Originally posted by mrphil
Hmm, try this one instead:

SlaveStoreCollection.xml

But make a backup first! It might remove all your recordings, I cant try this myself at the moment.

I'm having a problem with the importer, and I think it's the same one being discussed here. I posted a complete description of my problem here http://discuss2.snapstream.com/vb/showthread.php?s=&threadid=9248 and based on the reply there, I now believe it's caused by duplicate program IDs.

I have tried deleting ExceptionsRecordingCollection.xml and SlaveStoreCollection.xml, and then re-importing - but it did not seem to help at all.

Have you made any progress investigating this problem? Would it be possible to just generate a completely random UUID for each program (rather than base the ID on the program name), to avoid the "duplicate show" problem? Any other suggestions I can try?

Thanks a bunch for the importer!

Tim Francis

[Gman]

Originally posted by TimFrancis

I now believe it's caused by duplicate program IDs.


That does sound like it could be my problem. I have noticed that it always the shows that are on every day that are impacted by it. Also, deleting the ExceptionsRecordingCollection.xml file seems to fix the shows that are on multiple times a day (but not those that are on only once a day).

In the post you link to you talk about seeing "This show will not be recorded because it is a duplicate" in small writing somewhere, but I don't see anything like that. Where is it on the screen? You see it on the EPG, right?

[TimFrancis]

Originally posted by Gman


That does sound like it could be my problem. I have noticed that it always the shows that are on every day that are impacted by it. Also, deleting the ExceptionsRecordingCollection.xml file seems to fix the shows that are on multiple times a day (but not those that are on only once a day).

In the post you link to you talk about seeing "This show will not be recorded because it is a duplicate" in small writing somewhere, but I don't see anything like that. Where is it on the screen? You see it on the EPG, right?

Yes; I see it on the full screen EPG. I select the show in question (with the <crossed out> red 'N' ), and I see the text at the bottom of the screen, next to the description of the show.

[Christoph_vW]

I have fixed some bugs in xmltv...

-Now the correct path will be detected on non english windows
i.e. "C:\Dokumente und Einstellungen\All Users\Anwendungsdaten\" in german windows

-replaced the FExist code -> now the filesystem-object is not necessary anymore

-fixed a bug in the INI -code that could cause a crash if the file would be larger than 1000 lines

-replaced Variant Variables through the correct Datatype -> will be faster

and a lot of other things...

The new code (with binaries) is available here:
www.ApiViewer.de/downloadstore/importsourceNew.zip

[mrphil]

Thanks, I have changed the first post to include the new build.



Have you made any progress investigating this problem? Would it be possible to just generate a completely random UUID for each program (rather than base the ID on the program name), to avoid the "duplicate show" problem? Any other suggestions I can try?

Yes, this problem is caused by the epgid number being used when it has recently been used for another *recorded* show.

Ideally, the best way would be for the importer to store the epgid it uses for each show, and not reuse it again for at least 30 days. That way PVS would never not record it due to previous recordings.

Right now if you look in the database, you will see entries which have a title of "EPGID placeholder". The last 5 digits are the actual episode number, which on this entry is set to "00000", or episode 0. The importer then increments by one for each episode it finds which matches that show title, so 00001 00002 etc.

Maybe the best workaround if someone has time to change the code is once all data has been imported is to set the "EPGID Placeholder" number to the highest used number in this run. So, on the next import it will increment the numbers starting at the highest previously used number, thus no dupplications from the past. Once the episode numbers reach 40000, reset them back to 00000 and it can all start again.

I hope this makes sense to someone!

[PiperD]

I think I understand. I wrote a VB script to do just what you describe (until the functionality is implemented into the Importer) - I'll try an import and see what happens.

[Sandbag]

Hello everyone.

I am just trying to install the XMLTV but am stuck on step 4: Setup mrphil's XMLTV Importer.

I have extracted the files to c:\program files\snapstream\XMLTV, but cannot find register.bat anywhere.

I decided that I would leave that part out and ran the XMLTVImport program for the first time. I selected the uk.xml file and clicked done. However I just get an error message saying:

Run-time error: '9' subscript out of range

Has anyone else had this problem and know how to fix it?

Thanks

[huotg01]

Originally posted by Sandbag
Hello everyone.

I am just trying to install the XMLTV but am stuck on step 4: Setup mrphil's XMLTV Importer.

I have extracted the files to c:\program files\snapstream\XMLTV, but cannot find register.bat anywhere.

I decided that I would leave that part out and ran the XMLTVImport program for the first time. I selected the uk.xml file and clicked done. However I just get an error message saying:



Has anyone else had this problem and know how to fix it?

Thanks

I also got this exact same message at step 4. I was using _na

G.

[mrphil]

I have changed the build on the first page to the older version without chnages changes made by Christoph_vW, please download again and try importing.

[Sandbag]

Thanks. The old build works like a charm. I would also like to thank mrphil for your quick response and continuing work on this project. If it wasn't for you, us noobs wouldn't be able to use PVS to its full potential. Thank you.

[drsnorlax]

can someone help me, I'm trying to get the XMLTV importer to work and I'm not sure what country code to use, there doesn't seem to be a guide or list to tell me which code to use for Canada. I tried it with the settings that the guide on snapstream, but I don't get anything.

Thanks, and sorry if this is obvious and I'm just overlooking something, any help is greatly appreciated.

[huotg01]

...just replace the
_UK
used in the Percy's guide by
_na
for canadian (and US by the way) listings.

C:\XMLTV> xmltv tv_grab_uk --configure
replaced by
C:\XMLTV> xmltv tv_grab_na --configure

and
C:\XMLTV> xmltv tv_grab_uk --days 3 >uk.xml
replaced by
C:\XMLTV> xmltv tv_grab_na --days 3 >na.xml
for uploading 3 days of programmation

G.

[TimFrancis]

Originally posted by mrphil
Right now if you look in the database, you will see entries which have a title of "EPGID placeholder". The last 5 digits are the actual episode number, which on this entry is set to "00000", or episode 0. The importer then increments by one for each episode it finds which matches that show title, so 00001 00002 etc.

Another possible short term workaround idea, this one hopefully even simpler to implement... can you add an option to allow the initial episode number to be passed in as a parameter. This way I could change the starting point every day from my batch script, and so reduce the likelyhood of a conflict. In other words, instead of the first entry being set to "00000" every time, allow me to set the first number, and then the code could continue to increment for all other shows found, as now.

I don't have VB, or I'd try this change myself. If anyone can make the change, I'd really appreciate a copy ... PVS is rapidly becoming less useful for me, as more and more shows are starting to not record.

[jenandrew]

I have had mixed results with MrPhils importer. Maybe somebody can help me understand what's going on, and perhaps resolve my questions.

I have tried everything that I have understood to try on this thread, including restarting the import with a new copy of the database.

One thing that I have noticed is that the importer never finishes. I have left it for over a day, watching the Task Manager processor usage drop to zero, and still it doesn't finish. I do know that it is doing something, as I have used Access to look into the database and see my listings. Eventually I use the Close button on its window to exit...

I have also noticed, when in the SnapStream Program Guide, that there is a keystroke, <<U>> that will update the database. When I try that, it briefly 'works' and then comes back, with no change. I have a message across the top that the EPG data is low.

When I look at the SnapStream log, I see the following:
[9/16/2003 19:49:07] Starting clean update of database.
[9/16/2003 19:49:09] Database failed to insert new data into the temp tables while refreshing data.
[9/16/2003 19:49:10] Could Not Update Database. Failed at 127082189479060000 to 127083917479060000: Error 0x80004005
[9/16/2003 19:49:10] Clean update of database did not fully complete.

Anything I should be looking for?

Thanks in advance!

[cowtown]

Hey guys, thanks for all the help, I think I got the importer working well except one little thing. Now when I go into the guide to pick a program to record, all the listings are correct, but they appear to be in the wrong time band (IE 7:00am, versus 7:00pm). I set two programs to go to see what happens but I think it will probably just record the wrong program.

Any tips?

[norseman]

Hi jenandrew.

Have you disabled the Allow Automatic Update in Configure->Snapstream.Net Settings?
If you are using MrPhills importer, then you are NOT using a Snapstream.Net account, and the Allow Automatic Updates should be disabled.
Also the keystroke <U> is only for the Snapstrema.Net account.

Norseman

[jenandrew]

Yeah, thanks, I did disable that automatic update, and I suspected that the <U> key was for the Snapstream.net update. I just threw that in on the off chance that it would be germane in some unexpected way.

Thanks for taking the time to reply!

[cowtown]

Figured it out, when I was using the XMLTV automatic utility it was downloaded it for -700, then for whatever reason when the importer did it's job it was making my listings several hours out, so I tried it several different ways and ultimately using a +600 time difference in the importer got the listings to sync up.

Thanks again for the utility!

[Scott Anderson]

Good luck jenandrew. I have been having the same problem on and off for a few weeks and have been unable to fix it. Hopefully, someone smarter than us has seen this bug and figured out a way to squish it.

[kernunnos]

One thing that I have noticed is that the importer never finishes. I have left it for over a day, watching the Task Manager processor usage drop to zero, and still it doesn't finish.

I'm having the same problem here. It seems like the importer hits a snag when importing the listings output from xmltv and simply hangs (in mine, my EPG has no data beyond 7.30pm on Sep 30 when it should have imported all the way up to Oct 6).

This seems like a pretty serious problem and I hope the importer's developer will look into it.

[dodgechargerfan]

Where is the timezone information set?

I can successfully import listings but the times for the programs are all off.

I'm not sure the TZ is the problem, but it's the first logical thing to check, I think.

Once I know here it is, how do I know what to set it to.

I assume -5 means 5 hours behing GMT.
Is this correct?

Thanks.

[kernunnos]

I'm having the same problem here. It seems like the importer hits a snag when importing the listings output from xmltv and simply hangs (in mine, my EPG has no data beyond 7.30pm on Sep 30 when it should have imported all the way up to Oct 6).

To give some more information, here's what shows up in convert.rpt:

9/27/2003 6:03:41 AM Time zone =4
9/27/2003 6:03:41 AMC:\XMLTV\NA_SOR~2.XML
9/27/2003 6:03:41 AMC:\WINNT\Profiles\ALLUSE~1\APPLIC~1\SNAPST~1\PER SON~1\SS_PVS~1.MDB
9/27/2003 6:03:44 AMReading stations
9/27/2003 6:03:51 AMreading programme data
Runtime error occured: Type mismatch 13

[Scott Anderson]

dodgechargerfan,

What I recommend is to do a quick xmltv update and set the timezone offset in the importer to zero. Then bring up the FSUI of Snapstream and look at the Program Guide. Using common sense or a TV guide (whichever is easiest), figure the number of hours of error. ie. the six o'clock news shows as on at 11 am = 7 hours. Exit FSUI, enter the number of hours in the importer config, reupdate xmltv and importer and have fun.

[Scott Anderson]

Whenever someone says "it goes without saying"...it usually means that you need to say it.

Don't forget that whenever you make changes to importer config that you need to close and then restart the importer (to process the data) for the changes to be effective.

[dodgechargerfan]

Thanks for the info.

I found it in the xml files and manually edited it there.

I also set up the asp files so I can use the web admin to access the guide.

The web version is perfect.
The windowed version is different.

Bizarre.

Yeah, I learned quickly that I had to ext and restart PVS for most things that I change..

I'll keep pluggin away at it and eventually I'll figure it out.

Unfortunately, I'm not getting any audio in the recordings.
Tried all the stuff in the knowledge base - no luck.

[Scott Anderson]

Am I understanding you correctly...did you just say that you are using the xml data from xmltv importer and viewing it through the guide service in web admin?

How? I would love to be able to do that.

[RobertH]

check out the community page:
http://www.snapstream.com/Community/articles/progguide

[Scott Anderson]

After reading through the guide, I notice that it requires IIS (Internet Information Services) to be installed.

Unfortunately, IIS is not part of Windows XP Home (what I bought).

Corporate and Pro users (no matter where they got their software, wink wink) can use this asp to their hearts delight.

[ctebo]

I now am having problems with xmltvimport hanging / exiting on current xmltv files...

Its been working like a charm for the past month or two, but I can't get any data in past the 30th of september at about 7pm. The convert.rpt file will end with:

Runtime error occured: Type mismatch 13

I'll spend some time tonight at home trying to produce a minimal xml file that demonstrates this problem to upload here. If need be I'll install VB and do some tracing to determine where the crash is happening.

Anyone else running into this?

[Scott Anderson]

That's the problem I am (we are) experiencing. I hope you can figure this out as no one else seems to have any ideas.

[ctebo]

So I've tracked down the problem that is causing updates for data on the 30th of september to fail... The code has an error that is causing it to think that september has 31 days. This is of course not true, and causes the code to fail. I will post an updated version of the executable shortly.

For those with a coding bent, the error exists in the date_adjust function.

As well, having stepped through this code, it seems that there is a significant opportunity to improve performance with a few small changes - currently the code iterates through all program/episode data in the database for each show being processed. This can be done much more efficiently. I'll spend some time later tonight mucking about with that code and post it here as well.

[Scott Anderson]

ctebo,

You the man! Everybody knows "30 days hath September..."
mrphil (the original writer) has stopped doing coding improvements as he is focusing on other projects. He is quite receptive to others patching the code and subsequently posting the file for download.

Looking forward to your fixes.

[ctebo]

So here's my fix for the "31 days hath september" bug...

I've attempted to match the contents of the original zip file.

This zip file contains:
XMLTVImport\ImportXMLTV.exe - rebuilt with the fix
XMLTVImport\comdlg32.ocx - unchanged of course
XMLTVImport\readme.rtf - unchanged
XMLTVImport\register.bat - unchanged
XMLTVImport\importsource.zip - includes source code changes to correct this bug

To fix your existing installation, unzip the attached file and replace the ImportXMLTV.exe file that you have on your machine with the one included in the zip file.

If you run into any problems, post here. I'll follow up.

For a new installation, just treat this file as if it were the standard download.

Related to this, as enough people are using this code, it might be prudent to setup a sourceforge site to maintain revs of the code and to publish new builds as they become available. I'd be happy to do so. Any comments/thoughts on doing this MrPhil?

[kernunnos]

The fix for the bug works as advertised. Thanks ctebo.

A sourceforge site would bring the importer on par with xmltv in terms of project maintenance, so it's definitely a good idea. Hopefully it will entice people to start contributing (or even learn VB :)).

[Scott Anderson]

ctebo,

Thanks for the fix. It works very well. Since you were so good with that one, how about another.

I am having an issue with the importer rejecting programs that end at midnight.

convert.rpt :

29/09/2003 8:56:50 PMError occured with show:
29/09/2003 8:56:50 PMS: 20030929233500 -0700 E: ID: C12civi.zap2it.com N: The Tonight Show With Jay Leno

This happens for every show on any channel that the program ends at 2400 (or 0000 depending on how you look at it).
In the FSUI, the timeslot shows "Unavailable".

Is anyone else experiencing this? Is it some issue to do with the fact that the show ends on a different date than it began or something? Or is zap2it providing bum data?

HELP!

[mdorion]

Originally posted by Scott Anderson
I am having an issue with the importer rejecting programs that end at midnight.
HELP!

Try "sorting" the data using "xmltv tv_sort"

See: tv_sort

Sort listings into date order. Also takes the opportunity to add stop times to programmes missing them and sanity-check things like no overlap.


This is my batch update script:

xmltv tv_grab_na --days 8 > listings-presort.xml
xmltv tv_sort < listings-presort.xml > listings-sorted.xml
XMLTVImport\ImportXMLTV

Obviously, ImportXMLTV is configured to load listings-sorted.xml

[Scott Anderson]

Unfortunately, the tv_sort trick didn't work. Still having same problem with shows that end at midnight. Thought that was gonna fix it too.

Edit: Oops, I take that back...tv_sort fixed it, I am just an idiot and had a little error in my batch file.

[mrphil]

Originally posted by Scott Anderson

29/09/2003 8:56:50 PM S: 20030929233500 -0700 E: ID: C12civi.zap2it.com N: The Tonight Show With Jay Leno

For the record, to help you see whats going on here. In the log line you posted:

S: is start time retrieved for this show in the xmltv file
E: is the end time. In this case there is no end time, hence this show does not get added to the guide.
ID: is the channel identifier used in the xmltv file
N: is program name

[mrphil]

Originally posted by ctebo
So here's my fix for the "31 days hath september" bug...



Oops! :o Thanks for sorting that out. I must rhym more often! "30 days hath september....."

Related to this, as enough people are using this code, it might be prudent to setup a sourceforge site to maintain revs of the code and to publish new builds as they become available. I'd be happy to do so. Any comments/thoughts on doing this MrPhil?

That would be an excellent idea, please feel free to go ahead. This code only exists for the benefit of the community, so any additional work that makes things more stable and easy to use is a good thing.

Whilst I have little time to contribute right now, I can certainly answer any questions anyone has regarding how PVS works with its database, and what specific functions are for in the importer code.

[kernunnos]

I am having an issue with the importer rejecting programs that end at midnight.

convert.rpt :

29/09/2003 8:56:50 PMError occured with show:
29/09/2003 8:56:50 PMS: 20030929233500 -0700 E: ID: C12civi.zap2it.com N: The Tonight Show With Jay Leno


Note that tv_sort can only compute the end-time for a program by looking at the start-time for the next program on the same channel, and it only does this if the end-time is not already provided. So if zap2it does not provide the end-time for a program and it also happens to be the very last program for that channel in the listings you've grabbed, then tv_sort can't do anything about it.

I've always had the same error in my convert.rpt, but since I've always been grabbing 12 days worth of listings every 3 days, it's never been an issue.

[Scott Anderson]

Thanks for your input regarding the "end time" problem. The tv_sort routine fixed it up (except of course for the last day of listings).

For the record this is what my batch file does every night at midnight:

echo Beginning xmltv download of %1 day(s) of data
xmltv tv_grab_na --days %1 >presort.xml
echo Sorting Data to correct errors...
xmltv tv_sort < presort.xml > na.xml
C:\Progra~1\SnapSt~1\Person~1\distro\importxmltv.e xe
echo Finished update containing %1 days of data.

All I have to do is edit the shortcut to append the number of days of data I want it to download.

> update.bat 10

Would obviously get 10 days worth of data. Then simply adding the task to "scheduled tasks" has it run every night.

FYI: Zap2It seems to only provide about 11 or 12 days of data at any given time.

If anyone is interested...adding the command:

taskkill /f /im WTLPVSApp.exe

at the beginning of the batch file would shut down PVS
prior to doing the update. Google the taskkill command and it is available for download if your OS doesn't include it (WinXP Home).

If anyone has a better method of shutting down PVS (perhaps a little more gracefully) I am all ears.

[mrphil]

Originally posted by Scott Anderson

If anyone has a better method of shutting down PVS (perhaps a little more gracefully) I am all ears.

You only need to shutdown PVS if you are going to move, delete, or rename the database file. Updating the listings can happen with PVS running and should not have any adverse affects. I never shut down to update the listings.

[Scott Anderson]

Just in case you have been so wrapped around getting xmltv importer and Snapstream working together that you missed this update on canada.snapstream.net (like I did).

http://discuss2.snapstream.com/vb/showthread.php?s=&threadid=9105&perpage=15&pagenumber=5

Dang it...I can't believe that I missed this happening over the summer, and just when I finally got the xmltv thing completely working and happy.

[Splint]

As I do not have VB6 I've had to convert mrPhils Importer to Delphi 7 to try this out.

I start out but making a list of all the EPGIDs appearing in the SlaveStoreCollection.xml and make sure I don't reuse any of them while assigning EDGIDs to the episodes in the PVS database.

With this improvement I'm able to record all 3 episodes of Simpsons appearing on a specific channel every day so it seems to work.

I've also implemented automatically adding task into the task scheduler to run the importer at a specified time.

At start-up this importer doesn't start importing automatically. You have to press the import button. The program can be started with the /auto parameter to let it run silently and close when it is finished.

Thanks to mrphil for this great tool and its source. Congratulations to all of you lucky Canadians.

Attachment removed!

[Gman]

Originally posted by Splint

With this improvement I'm able to record all 3 episodes of Simpsons appearing on a specific channel every day so it seems to work.


Splint, if this works for me as well, you'll here me cheering all the way over there in Denmark. And I'm in NZ, so that's a hell of a long way.

[commy]

When I press the "OK" button in the Configureation window I get and error message (Access violation) and when I start the import the program seems to "hang" after the first channel.

If you could release the source code I would be more than happy to help you develop this tool.

PS: Im from Denmark too and have been programming Delphi for a couple of years so it could work out very well.

[Gman]

I'll just add a "ditto" to commy's experiences. Access violation, then a hang.

[Scott Anderson]

The source code is included in the download.

[Gman]

Splint didn't include source in his Delphi version download.

[Scott Anderson]

My bad, thought we were talking about the VB6 original.

[Splint]

I'll try to remove some code tonight that creates a task in the task scheduler tonight and upload it again. It's of no use at the moment anyway (but it will be later).

I plan on releasing the source when I'm done whith the most important improvements I plan to integrate into the program.

I'm currently working on integrating XMLTV so all configuration of it can be done from a configuration window.

[Scott Anderson]

Are you planning to add the XMLTV code to the importer in this process? If so, is that very wise given the constant improvements being made to XMLTV via sourceforge?

Pardon me if I misunderstand.

[Splint]

I plan to use XMLTV without the end user ever seeing the dos promt on the screen.
Technically I redirect what you would see in the dos promt to a file that I'm able to read. It is also possible for me to answer "yes" and "all" during the configuration process. I've used most of my weekend to figure the code needed out.
I've no intension of looking into the Java code in the XMLTV project nor modifying it in any way. Its great as it is. Just to include the newest XMLTV release with this release. I think the myHTPC projects importer does it likewise.

[commy]

Do you have an ETA when you will release the source? or have a fully working version ready?

Im getting tired of missing episodes because of the "duplicates bugs" in the original XMLTV importer util :-(

[ctebo]

Sorry if I've missed this in earlier postings in this thread, but what is the "duplicates bug" I don't seem to have run into it here...

[Splint]

I've tried to remove some code. Please report back if it works. Otherwise I'll PM you (Commy) the sorce so you can help me with this issue :)

Attachment removed! Doesn't work anyway :(

[Splint]

TimFrancis mentions this in the top post on page 6 in this thread :)

[commy]

The error messages is now gone but it still hangs after the first channel number is entered in the "New station found" dialog.

[Splint]

Please read your PM.

[Splint]

Ok. Stupid bugs should be out of the way now. This release is tested by commy and me so enjoy over there in NZ :)

[commy]

thumbs up! for this great new tool!

keep up the good work Splint.

[root2121]

Hi Splint,

tried running your new delphi version but I am getting the error noted below. Basically I was using the VB6 version (which worked fine) and I installed the delphi version and pointed to my xml listings file and clicked on the import button. Do I have to go through the channel setup again? Is there anything specific I might need to do for setup? Just thought maybe the error below might be related to a setup issue. Any ideas? Thanks for your work on this and this would be awesome if I can get this going as I have been plagued with the duplicate problem for a while now.

Here is the error:

[10/7/2003 6:54:58 PM] Completed checking for new channels. 0 New channels found
[10/7/2003 6:54:59 PM] Processing guide data...
[10/7/2003 6:54:59 PM] Reading programme data
[10/7/2003 6:55:19 PM] Runtime error occured in Import: Element does not contain a single text node
[10/7/2003 6:55:19 PM] XMLTV to PVS3 EPG Importer v1.15 ended!

[commy]

try using a clean database (use a copy of the mdb.clean file)

the delphi port uses new episode GIDs so it might get confused with the old data in your database.

[Gman]

Originally posted by Splint
Ok. Stupid bugs should be out of the way now. This release is tested by commy and me so enjoy over there in NZ :)

No joy, I'm afraid.

No Access Violation (although I didn't go into the config this time) but the import got past the Checking for new channels then hung on the "Processing Data" phase.

[Splint]

root2121:
Your xml source includes one or more shows, that does not include either start time, title or channel number. I accidently assumed this info was awailable for all shows... my bad.

Gman:
If this release doesn't help please post some of the last lines in the convert.rpt file. This file is filled up with all the debug info a developer could ever dream of :). Then it's easier for me to find the trubled area.

In general:
The importer uses the same config files, debug file and database structure as mrphils importer did. There is no reason to run the config part again if the old version were correctly setup.

[Splint]

An error in the last build caused no titles of programs to be read from the xml file. Stupid error... sorry but 13 hours at work today was a little long ;)

I now also seem to have stumbled upon the error of Gman with nothing happening. I'll look into it.

[Splint]

It took me 14 minutes to import 5861 programs so try to give it some more time Gman. Try this program out.

[dleewo]

I noticed other had this problem, but I don't see a resolution in the forum.

When I run the utility for the first time, it reports a new channel found and asks for a callsign and a channel number. Regardless of what I enter, pressing "ok" keeps me in the dialog. I have to kill the app from the task manager.

Is there a workaround for this? Am I using the utility wrong?

[Splint]

For the time beeing you can just close the new channel dialog using ALT+F4 and the channel is saved anyways with the name and number you specified. I've corrected this now but I can't make a release at the moment. I asume you are using the 117 release.

Thanks.

[ctebo]

I've made some changes to Mr Phils import tool that include a fix for duplicate records (a la splints delphi tool) along with performance improvements and have dubbed the new version of the tool 1.10.

Duplicates are corrected by inspecting the "SlaveStoreCollection.xml" file maintained by PVS. This includes entries for each program that has been recorded and maintained in your Video Library. I have made the assumption that this file is sitting in the same directory as your snapstream program guide database file, rather than somewhere else. If anyone is running the import tool to populate a database other than the one snapstream is using, let me know and I can add a configuration entry to fully specify the path for the slavestorecollection.xml file.

The performance improvements come from tweaking the database queries made from the tool. Not a significant coding change, but in my environment, processing 25,000 programs (about 10 days of data for me) now takes 15 minutes with this new build as compared to 40 minutes with the previous one.

I will be setting up a sourceforge site over the next day or so to provide a single point for retrieving current and past builds of the tool, along with the source.

To upgrade an existing installation, open the attached zip file and replace the current version of ImportXMLTV.exe on your machine with the new one included in the zip file.

To perform a new install, follow the instructions provided in the first post of this thread.

Let me know if anyone runs into issues...

[Per]

I just downloaded the new version. It starts OK but after checking for new channels nothing happens. "Processing guide data" is the last message.

The app isn't hung or anything, I can stop it by clicking finnish, but first after cklicking config and exiting that window.

My convert .rpt looks like this:
2003-10-15 08:50:42 Time zone =-2
2003-10-15 08:50:42C:\UTILS\ss_xml\listings.xml
2003-10-15 08:50:42C:\DOCUME~1\ALLUSE~1\APPLIC~1\SNAPST~1\PER SON~1\SS_PVS~2.MDB
2003-10-15 08:50:42 Read 1 EPGIDS from SlaveStoreCollection.xml
2003-10-15 08:50:42Reading stations
2003-10-15 08:51:38reading programme data
Runtime error occured: No current record. 3021

Any ideas?

/Per

[ctebo]

Per and all...

Sorry. My bad. A bug exists in 1.10 when running against a clean PVS database that causes the tool to fail. The "clean" database I was using for testing wasn't completely clean.

So a fixed build is attached. To upgrade an existing installation, open the attached zip file and replace the current version of ImportXMLTV.exe on your machine with the new one included in the zip file.

To perform a new install, follow the instructions provided in the first post of this thread.

Chris

[Adrian]

Great tool - Thanks.

Have you thought about extending this to include XML listings from digiguide? This is the UK guide and keeps 14days listings. The XMLTV feed only gives us 4 days.

http://www.digiguide.co.uk
________
KTM 525EXC-G (http://www.cyclechaos.com/wiki/KTM_525EXC-G)

[mrphil]

Originally posted by Adrian
Great tool - Thanks.

Have you thought about extending this to include XML listings from digiguide? This is the UK guide and keeps 14days listings. The XMLTV feed only gives us 4 days.

http://www.digiguide.co.uk

It is impossible to extract 14 days worth of listings from digiguide, you can at most extract about 7 programs in every 24 hours. Their files are not XML, they are encrypted, and have said several times on their forums that even if someone were to crack the encryption, they can change it in minutes. Sorry.

[ctebo]

I have setup a website (through SourceForge) to host binaries and source code for the VB import tool that Mr Phil was kind enough to put together for us.

Visit: http://pvsxmlimport.sourceforge.net to get updates on new builds as they become available.

For those who wish to contribute code changes, I'm moving the source code into the sourceforge hosted CVS repository as we speak, and will open that up to any and all who wish to make improvements.

As for upcoming additions to the code, I'm working on having Recording Manager Jobs be maintained after you've cleared the PVS EPG database. Any of you who have spent some time tweaking your recording jobs like I have, only to discover that all of that is lost when the database is cleared, should have some relief on this front shortly.

Chris

[Per]

I've been using the importer since the start, and now it's really usefull, thanks to everyone who's contributed.

I few things though:
It looks like the show title is also imported to the episode title. My recorded shows always get the name ' Simpsons - "Simpsons" ' in my video library. No biggie, but I'd like to save some bits ;-)

The second thing. One station shows the Simpson's twice a day. After the import I have to manually remove the re-runs. However, after the next import ALL shows are enabled for recording again.

Any ideas?

[ctebo]

Hi Per,

The program title's and descriptions are pulled from the xmltv output file, and should include the title and description if they are available? Which xmltv grabber are you running? I use tv_grab_na and get descriptions for most of my episodes. If you let me know which grabber you're running, and what xmltv configuration data you provide, I can pull the same files you do and see if the descriptions are lurking somewhere else in the xmltv output.

As for stations that broadcast the same show multiple times per day... I've toyed with several approaches to this issue. Ideally, if a program is rebroadcast, we shouldn't have our record all jobs record the program multiple times. My thinking on this is to have the code flag all occurances of the same show appearing on a single day and on a single channel, with the same title and description to be flagged as duplicates. That way only the first episode would be recorded. This may cause some problems though. For example, syndicated shows usually don't have episode specific descriptions, and broadcasters often run two different half hour episodes back to back. If their descriptions don't differ, then by the approach I've described, only the first of the back to back episodes would be recordable. Back to back episodes could then be handled as a special case. I've hacked together a solution to this approach on my end that doesn't quite work right yet, but am going to tinker with it some more. One side note is that it makes the import process run alot longer. If it ends up working out well for me, I'll consider merging it into the importer code and allow users to enable and disable this option at their own choosing.

Thoughts anyone?

ctebo

[Per]

I'm using the tv_grab_sn. It provides show name and description, but no episode name. After the import is done and I check the database the episode name is the same as the show name, hence the ' simpsons - "simpsons" ' in my video library. My conclusion is that the importer puts the show name in the episode name field if it don't find an episode name in the xml-file.

As for the same episode multiple times a day. In my part of the world usually the description for the re-run is just that, "Re-run from earlier today"

I'n my case it would be easy, if a certain string appears in the descripton it's a re-run.

[stere0type]

is there any support for the "xmltv importer"??
or email adddress?

my time zone is -3:30
and all I can input is 1,2,3,4,etc.. can't input 3:30.


any idea or help?
thanks

[ctebo]

Stereotype,

As it stands right now, the importer only supports time-zone offsets of whole hours. I will add support for your case in the next version (1.12). That version should be available by mid next week.

Just so I can run a test case with your actual data to make sure all is well, which xmltv grabber are you using (eg. tv_grab_na) and what config information are you passing to xmltv to get listings for your area (eg. postal code).

[stere0type]

Thanks can't wait to use the new version

here's my info
tv_grab_na
postal code: A1W1N5

I'm located in
Newfoundland, Canada

[bast]

Hi,.
i am using the new importer and have noticed that everyday from 11pm to 12pm the programs are "not available." WHen i import using the original importer, all the data is there, so the issue is not the xmltv data. FOr some reason, the importer is not filling in this one timeslot. Any suggestions?

[ctebo]

Hi Bast,

Your problem can be solved by using the xmltv sort function. Basically the problem is that the data provider isn't including finish times for programmes at the end of the day.

The "xmltv tv_sort" command will correct this data for you. My xmltv batch file works as follows...

xmltv tv_grab_na --days 10 > listings-nosort.xml
xmltv tv_sort < listings-nosort.xml > listings.xml
XMLTVImport\ImportXMLTV

And ImportXMLTV is configured to read the data from listings.xml

[bast]

Thanks ctebo. That sorting thing did the trick.

[ctebo]

Version 1.12 of the import tool is now available from:
http://pvsxmlimport.sourceforge.net

To upgrade an existing installation, merely replace your installed copy of ImportXMLTV.exe with the one included in the 1.12 zip file.

New installations should follow the installation instructions provided in the first post of this thread.

Changes with this release include:

- Added support for time-zone offsets that include minutes as well as hours. Existing config files will default to 00 minutes. The configuration window will allow a user to specify minutes as well as hours. This was specifically requested by a user in Newfoundland where the offset is 3:30.

- When the pvs epg database is cleared (by replacing it with ss_pvs_db.mdb.clean), the importer will now preserve existing Recording Manager Jobs as recorded in the JobCollection.xml file.

- The log entries displayed in the importer UI are now scrollable both horizontally and vertically, so that all content presented there can be viewed.

- Progress updates are logged in the importer UI, so that during long imports, users can get some visible feedback that the process is still running.

[ctebo]

I'm thinking about implementing changes that could be made to make the xmltv import tool more useful for users out there.

I've been making changes that address some of the problems that I've run into over the past couple of months while using the tool, but grant that I'm not the typical user - I'm happy to write code and munge data in databases to get things working.

As I've setup my girlfriend with a Snapstream system (using the import tool for guide data) over the past month, there are several features that seem to be needed with the import tool to make this solution wife/girlfriend/mother friendly.

- Installer - A standard windows installer that installs the import tool, along with a version of XMLTV to perform the imports.

- Configuration - The import tool configuration should provide a windows gui based configuation tool for both the import tool and xmltv.

- Updates - The import tool should provide an optional means to automatically update both the import tool and xmltv based on the availability of new updates. Getting new builds of xmltv installed is a more pressing requirement on this front.

- Scheduling - Configuration of the import tool should automatically schedule xmltv runs to grab listings and the import of those listings. This should provide means to schedule updates based on the availablity of data from the xmltv data provider.

- Channels - Cleaner handling of channels and their callsigns. Updating the default lineup automatically without having to dig into the web ui to get your channel callsigns/names to show up.

I'm getting ready to implement a new version (should we call it 2.0) of the import tool that does all of these things.

If you have any other features/requirements that you would appreciate, I'd love to hear about them. If they can be implemented against the current import codebase, I'd be happy to tackle them first.

Share your comments here!

[mrphil]

Ctebo,

A while back I was going to add a couple of other requested features, you may want to add these to the list:

Ability to import CSV files as well as XML.

Ability to load all files in a specified directory instead of just one file; some xmltv grabbers output every day in a seperate file named by date.

Good work so far!

[Per]

Entries that don't import correctly: is it possible to save those to a separate file? Then you can check what's wrong.

An other feature is to handle the weird thing my XMLTV source that sometimes lists a major programme and then sub-shows, PVS isn't really good at taking care of that.

/Per

[ctebo]

Per,

I've had some time to look at the output of tv_grab_sn, and it seems that this grabber doesn't get as much data about a program as others do...

For most shows that I grab using tv_grab_na, I end up getting a title, sub-title, and description. These map to the program name, the episode name and the episode description respectively.

As an example, tv_grab_na spits out this for an episode of the simpsons:

<title>The Simpsons</title>
<sub-title>Treehouse of Horror X</sub-title>
<desc>Homer causes catastrophe at the plant; Bart and Lisa X-ray Halloween candy; the family murders Ned.</desc>

In the data I pulled with tv_grab_sn, I can't find a single program with a sub-title entry. They all look like:

<title>Simpsons</title>
<desc>Amerikansk animerad serie från 2002-2003. Även i natt.</desc>

As you suspected, the importer is coded to set the episode name to the program name if one isn't found in the xml file. database.

Until dagenstv.com starts providing episode descriptions, there's not much that can be done on this front.

[ctebo]

MrPhil and Per, thanks for the responses. A question for each of you though...

MrPhil,
CSV files should be straightforward to implement, but I'd need a pointer to sources for guide data that can be found in csv format. Any ideas where I can find some?

Per,
Can you point out some shows that demonstrate the sub-shows in your XMLTV data?

[Per]

Here's a sample from my listings.xml showing my issue with sub-shows.

[Splint]

In the new release of the xmltv project (0.5.20) a new filter has been added. This solves your problem Per.

From https://sourceforge.net/projects/xmltv/:
A new filter tv_remove_some_overlapping gets rid of the 'umbrella' programmes that appear in some listings sources, programmes whose only purpose is to contain others.

[ctebo]

A request for your comments on a feature I'm working on including in an upcoming build of the importer...

With the changes in a previous release of the importer, we no longer have the problems with shows not being recorded because they are seen by PVS as duplicates of an already recorded episode. But with that problem now out of the way, I'm discovering some problems in the other direction - too many episodes of a show are being recorded.

Some examples probably make this the clearest...

In my region, the XMLTV output identifies a show called "Global News" being broadcast on channel 3 each weekday at 12 noon, 5:30pm and 6:00pm. The XMLTV data does not include any further episode detail. I know that each of these 3 daily broadcasts includes different content. I want to record the 6pm broadcast using a "Record All" job. The problem is that all 3 of the daily episodes are recorded, not just the one at 6pm.

As well, some of my cable channels only run 6 hours of new broadcasts a day. They then re-broadcast this content 4 times over a 24 hour period to provide a full day's worth of programming. If I choose to "Record All" of one of their shows, I wind up with 4 copies of one episode.

I've solved this problem on my end with a quick hack that goes through the XMLTV output data renaming shows based on some rules I specify. I'm going to introduce this functionality to the XMLTV Importer, and would like some feedback on the types of rules that would be useful for these types of problems that you are running into with your recordings.

The following rule type is effectively what I'm using to solve this problem on my data: The rule specifies a channel, a time, and a program name, as well as a replacement program name. When a program is being imported into the pvs database, if the channel, time and program name match one of the rules, then the program name recorded in the pvs database becomes the replacement program name that was provided with the rule.

So to solve the first example, my rule specifies channel 3, 6pm, "Global News" and provides the replacement name of "Global News At 6".

To solve the second example, my rule specifies channel 32, 7pm, "Open Mike with Mike Bullard" and provides the replacement name of "Open Mike with Mike Bullard - New Broadcast".

Would providing just this single rule type help most of you with these types of issues? Or are there other cases that I just haven't run into yet? There are lots of variations that i can imagine (day of week, rename shows not matching time, etc), but would like to minimize the number of rules that are supported in a first effort.

Thanks for the feedback,
ctebo

[Gman]

(Copying this here in case no one's monitoring the "Bugs" page of the Sourceforge project)

Runtime error 380
Whilst running v1.12 on Friday, the importer successfully
ran for a while (imported about 20% of the data) and
then stopped - reporting run-time error 380.

I searched for debug info, e.g. in convert.rpt, but could
not find anything useful.

Note that I re-ran the import using the "old version"
(v1.04, I think) which imported 100% of the data
successfuly.

[ctebo]

Hi Gman,

I saw your bug report on sourceforge. Any chance that you could forward me the xml file that generated the error? That should help to track down the error.

pm me to pass the file along through email, or post it here if you prefer

ctebo

[mangiov]

Hi,
i am a new user of Snapstream and since I live in Italy I have been using PVSimport to load my XMLTV program guide.
First of all, I'd like to thank everybody for the support I could get just by browsing the board and especially to the people that are working on PVS import.

I am dealing with a couple of issues on which I could not find any answer and that might be minor bugs of this wonderful tool.

The XMLTV parser really doesn't work well for italy since it uses a provider whose program guides are not complete for all the channels i need (e.g SKY italy or some terrestrial feeds); so I decided I was going to write my own parser following the XMLTV dtd.
Last night I tested the first alpha of my program and the XML generated would not load. I noticed that the <category> tag was placed before the <desc> tag within the <programme> tag. Just flipping the order of the tags fixed the issue (now <desc> comes before <category>. My question is: is this correct and if it is, what is the order in which I should place the other tags (credits, ratings, etc)?

Second issue is that, looking at the PVSimport code, I noticed that the when adjusting the date/time February is only checked for 28 days; 2004 will have 29 days in February :)

Thanks and sorry for the lenght of my message.

Vinny

[HEADRAT]

Are there supposed to be channel logos, if I look in the uk.xml file there are links to channel logos but they don't appear in the EPG ?

http://www.ananova.com/tv_listings/images/logos/logo-151.gif

Any ideas ?

HEADRAT

[KevinC]

" Runtime error occured: The Microsoft Jet database engine cannot find the input table or query 'update_table'. Make sure it exists and that its name is spelled correctly. 3078" (full copy below)

I've just installed 3.4 beta and am now getting this error when I run ImportXMLTV. Has the new beta broken the importer or am I the only one getting this?

KC

[12/11/2003 23:19:03] XMLTV to PVS3.0 EPG importer.
[12/11/2003 23:19:03] Starting run at 12/11/2003 23:19:03
[12/11/2003 23:19:03] --------------------------------------------
[12/11/2003 23:19:03] XML filename is: C:\xmltv\nl.xml
[12/11/2003 23:19:03] EPG database file is: c:\documents and settings\all users\application data\snapstream\personal video station\ss_pvs_db.mdb
[12/11/2003 23:19:03] Video Library file is: C:\DOCUMENTS AND SETTINGS\ALL USERS\APPLICATION DATA\SNAPSTREAM\PERSONAL VIDEO STATION\SlaveStoreCollection.xml
[12/11/2003 23:19:03] Recording Manager file is: C:\DOCUMENTS AND SETTINGS\ALL USERS\APPLICATION DATA\SNAPSTREAM\PERSONAL VIDEO STATION\JobCollection.xml
[12/11/2003 23:19:03] Time zone adjustment is: -1:00
[12/11/2003 23:19:03] Clearing old data from EPG
[12/11/2003 23:19:03] Runtime error occured: The Microsoft Jet database engine cannot find the input table or query 'update_table'. Make sure it exists and that its name is spelled correctly. 3078

[ctebo]

headrat,

Most xmltv files do contain channel logos, but currently pvs doesn't support the use of logos in the epg. If the day comes where ss starts using channel logos, I'll do my best to get the xmltv logo data into snapstream.

Chris

[HEADRAT]

Thanks very much for your input, this afternoon I had a blank PC, this evening a fully functioning PVS box, I'm happy :)

HEADRAT

[ctebo]

mangiov,

Good catch on the date handling for February... I'll get that one taken care of before it stomps on us next february!

As for you efforts to import your own data, the xmltv dtd specifies the order of entries within the program tags. So that forces you to produce the entries in the appropriate order. If you look at the xmltv.dtd file (which is in your xmltv installation directory) you can pull out the order in which the tags are required.

What I'd be more interested in knowing is where you're getting the data to supplement the xmltv listings for your area. I'm working on some features to do that type of thing, and would love to know whether you've found another automated source for the data, or if you're manually assembling the data files.

Chris

[mangiov]

ctebo,
thanks for your clarification on my xml.dtd question. I wrongly assumed that since the xml is tag based, there wouldn't be such an issue.

As far as my program, I am basically going out to a website (www.personaltv.it (http://www.personaltv.it) ) and parsing the html code to generate the xmltv compliant file.

It's not much different from what the "official" italian xmltv parser does, only with a different website with more detailed data and with the complete channel lineup available in Italy.

If it could be interesting for you I am available to give you as much information/support as you need.

Vinny

[DaveG]

FYI, I am having the same problem with XMLTV Importer since installing SS 3.4 Beta.

I get an xmltvread info box with the following info :-

Run-time error '3078':

The Microsoft Jet database engine cannot find the input table or query 'update_table'. Make sure it exists and that its name is spelled correctly."

PVS XML Import window reads:-

[13/11/2003 21:30:06] XMLTV to PVS3.0 EPG importer.
[13/11/2003 21:30:06] Starting run at 13/11/2003 21:30:06
[13/11/2003 21:30:06] --------------------------------------------
[13/11/2003 21:30:06] XML filename is: C:\xmltv\xmltv-0.5.20-win32\uk.xml
[13/11/2003 21:30:06] EPG database file is: c:\documents and settings\all users\application data\snapstream\personal video station\ss_pvs_db.mdb
[13/11/2003 21:30:06] Video Library file is: C:\DOCUMENTS AND SETTINGS\ALL USERS\APPLICATION DATA\SNAPSTREAM\PERSONAL VIDEO STATION\SlaveStoreCollection.xml
[13/11/2003 21:30:06] Recording Manager file is: C:\DOCUMENTS AND SETTINGS\ALL USERS\APPLICATION DATA\SNAPSTREAM\PERSONAL VIDEO STATION\JobCollection.xml
[13/11/2003 21:30:06] Time zone adjustment is: 0:00
[13/11/2003 21:30:06] Clearing old data from EPG
[13/11/2003 21:30:06] Runtime error occured: The Microsoft Jet database engine cannot find the input table or query 'update_table'. Make sure it exists and that its name is spelled correctly. 3078

Is this a problem with SS 3.4 Beta?

[ctebo]

For those of you posting here regarding import issues and beta versions of snapstream, please visit this thread in the beta site:

[http://discuss2.snapstream.com/vb/showthread.php?s=&threadid=10398

[KevinC]

Thanks ctebo. I hadn't spotted the other thread.

Kevin

[GPKMan®]

First time posting, have been reading the threads for some time now.

Other than the XMLTV sources that exist now, can I import listings from www.tvguide.com? They have my cable company in the Caribbean listed.

Secondly, can I simply create a epg from scratch using existing channels from other localities, my cable company transmits standard US channels, usually from the East Coast.

Thanks for the help, I'm looking forward to getting this live!!

GPKMan®

[ctebo]

A new version of the importer is now available at PVS XML Import (http://pvsxmlimport.sourceforge.net)... This includes several significant changes:

1) It can support both the PVS 3.3 data schema, as well as the 3.4 beta schema. If you upgrade to this version of the importer, you'll be able to continue to generate guide data after the 3.4 version of snapstream is released (unless changes are made to the 3.4 beta schema prior to release).

2) An installer version has been introduced. This will make it simpler for you to install the importer, as well as deal with upgrades to future versions of the importer.

3) Configuration and imports are now managed through two different applications. If you run the installer version of 1.14, you'll find a "PVS XML Importer" group under your start menu's programs section. Within it you'll find "Configure PVS Importer" and "Run PVS Importer" as options. If you have setup scripts to automatically trigger the importer, be sure to update the path that they use to reference the importer - by default "C:\Program Files\PVS XML Import\ImportXMLTV.exe"

4) Channel lineup edits can now only be made through the configuration tool. This allows you to easily go back to a single screen to view the set of channels included in your xmltv files and edit them to provide the appropriate data for your region. If an import is executed using the ImportXMLTV.exe process, and channel changes are identified in the xmltv file, processing will abort.

5) The code has been significantly restructured. The benefits of this will be longer term. But most importantly will provide for a more maintainable code base, and allow some of the changes that will allow region specific funtionality to be more easily introduced.

6) Progress updates that are logged in the importer UI, are now logged to XMLTVImport.log as well.

To download the latest version, visit PVS XML Import (http://pvsxmlimport.sourceforge.net)

New versions of the installation instructions should be available in the next few days. But given that you brave souls have been able to get the current version up and running, I'd suspect that you'll have no problem getting this install in place.

As always, post your comments, criticisms and concerns here...

Chris Tebo

[ctebo]

GPKMan,

Unfortunately, I don't think that there are any xmltv grabbers for tvguide.com.

So, your best bet would be to grab listings using tv_grab_na (the north american grabber) and then configuring the xmltv importer to map the north american channels that you receive to the appropriate snapstream channels.

Not an ideal solution, but a start...

Chris

[GPKMan®]

Thanks Ctebo,

I have already started the testing of that concept, sounds workable to me, once I can do that channel mapping one time and do not have to re-map with every single import.

I'll keep you informed via this media.

GPKMan®

[Peter Pan]

If I try to 'edit channel Lineup' i get an error message:


Run-time error '339'
Component 'comctl32.ocx' or one of its dependencies not
correctly registered: a file is missing or invalid.

But I have registered 'comctl32.ocx'

I have WindowsXP SP1 installed. And I use the smaller zip-version of your software (Version 1.14)

[ctebo]

Rather than use the zip version of 1.14, can you use the installer version? It includes other dependencies that are required by these controls.

[Rene2003]

I'm using the new 1.14 version and PVS 3.4

I've configured everything, and the import runs OK. It identifies the 3.4 database.

However, when I go to the program guide it says "The program guide was unable to load any data. Please make sure your default channel line-up is compatible with netstream.net and restart"

Any idea's?

ps: Thank you for making this great tool. I use it in the Netherlands.

[ctebo]

Given that you're using the 3.4 beta?

Please visit this thread
http://discuss2.snapstream.com/vb/showthread.php?s=&threadid=10398 in the beta discussion forums.

I'll post the solution to your problem there.

Chris

[chntlychgr2006]

can someone help me? i used xmltv to get the listings, but when PVS XML Import converts it for snapstream, I load up snapstream, but nothing is shown besides "unavailible". I had set up everything properly (the MS access database, XML file linked in XML Import, Channel Name and Call letters setup correct in PVS), but something is going wrong in here.

[ctebo]

Could you provide some more details on the problem? We'll get this straightened out for ya...

To start, which version of snapstream are you running. What version of the importer? And what version of xmltv?

Could you provide the importer log data? If you're running 1.14 of the importer, you can find this data in C:\Program Files\PVS XML Import\XMLTVImport.log

If you're running a previous version of the importer, then copy the contents out of the Import application's UI.

Chris

[chntlychgr2006]

importer - v1.14
snapstream - v3.3 build 714
xmltv - v0.5.23

heres the log:
[11/26/2003 5:23:17 PM] ---------------------------------------------------------
[11/26/2003 5:23:17 PM] XMLTV to PVS3.0 EPG importer.
[11/26/2003 5:23:17 PM] ---------------------------------------------------------
[11/26/2003 5:23:17 PM] Starting run at 11/26/2003 5:23:17 PM
[11/26/2003 5:23:17 PM] XML filename is: C:\Documents and Settings\Administrator\dc.xml
[11/26/2003 5:23:17 PM] EPG database file is: C:\Documents and Settings\All Users\Application Data\SnapStream\Personal Video Station\SS_PVS_DB.mdb
[11/26/2003 5:23:17 PM] Video Library file is: C:\DOCUMENTS AND SETTINGS\ALL USERS\APPLICATION DATA\SNAPSTREAM\PERSONAL VIDEO STATION\SlaveStoreCollection.xml
[11/26/2003 5:23:17 PM] Recording Manager file is: C:\DOCUMENTS AND SETTINGS\ALL USERS\APPLICATION DATA\SNAPSTREAM\PERSONAL VIDEO STATION\JobCollection.xml
[11/26/2003 5:23:17 PM] Channel Lineup file is: C:\DOCUMENTS AND SETTINGS\ALL USERS\APPLICATION DATA\SNAPSTREAM\PERSONAL VIDEO STATION\LineupCollection.xml
[11/26/2003 5:23:17 PM] Time zone adjustment is: -5:00
[11/26/2003 5:23:17 PM] EPG database identified as version 33
[11/26/2003 5:23:17 PM] Clearing old data from EPG
[11/26/2003 5:23:17 PM] Retrieving EPGIDs for shows in your Video Library
[11/26/2003 5:23:17 PM] Found 0 EPGIDS in your Video Library
[11/26/2003 5:23:17 PM] Validating EPGIDs for Recording Manager Jobs
[11/26/2003 5:23:17 PM] Checking for new channels in XML file
[11/26/2003 5:23:20 PM] Updating the default channel lineup
[11/26/2003 5:23:20 PM] Creating list of current stations
[11/26/2003 5:23:20 PM] Completed checking for new channels. 0 new channels found
[11/26/2003 5:23:20 PM] Processing guide data...
[11/26/2003 5:23:26 PM] 500 listings processed.
[11/26/2003 5:23:32 PM] 1000 listings processed.
[11/26/2003 5:23:38 PM] 1500 listings processed.
[11/26/2003 5:23:40 PM] Completed processing guide data. 1636 programmes added to EPG
[11/26/2003 5:23:40 PM] Generating search data...
[11/26/2003 5:23:40 PM] Setting last updated time in database..
[11/26/2003 5:23:40 PM] Completed processing XMLTV file

[ctebo]

Based on your import log, it appears that all processing was done correctly. I then suspect that this means that there is an issue with the date/time format within your xmltv file, causing the data to be successfully imported, but for the wrong date.

Could you take a quick look into your XMLTV output file to verify that the date format is correct? To do this, open up your xmltv output file and search for "programme" to find the first program entry. You should then see something like:

<programme start="20041123053000 BST" stop="20041123060000 BST" channel="carlton.com" clumpidx="0/1">

The start entry indicates the start time for the show. This entry is defined (by XMLTV) to be in the YYYYMMDDHHmmSS format.

Let me know if this is the format you see in your file. As well, could you either post a small xmltv output file here, or provide the configuration data that you specify to your grabber (as well as which tv_grab_*** grabber you're using).

Chris

[chntlychgr2006]

heres a copy of what XMLTV outputed:

- <programme start="20031126003700 -0500" stop="20031126013600 -0500" channel="C4wrc.zap2it.com" clumpidx="0/1">
<title>Late Night With Conan O'Brien</title>
<desc>Director Ron Howard; musician John Tesh; the Strokes perform.</desc>
<category>Talk</category>
<category>Tabloid</category>
- <audio>
<stereo>stereo</stereo>
</audio>
<subtitles type="teletext" />
- <rating system="VCHIP">
<value>14</value>
</rating>
</programme>

is the "BST" in your post supposed to be "-0500" in mine?


heres what I have in the .CONF file that xmltv uses:
# config file: tv_grab_na 3.20031101
#
# this file is generated by running tv_grab_na --configure
# the only change you should make is prefixing 'channel:' lines
# with a 'not ' to signal that they should be ignored during the
# grab step
#
retry limit: 10
retry delay: 20
zip code: 20151
provider: 141459 Local Broadcast Listings
not channel: 2 WMAR
not channel: 3 WHSV
channel: 4 WRC
channel: 5 WTTG
channel: 7 WJLA
channel: 9 WUSA
not channel: 11 WBAL
not channel: 13 WJZ
channel: 14 WFDC
channel: 20 WDCA
not channel: 22 WMPT
not channel: 24 WUTB
not channel: 25 WHAG
channel: 26 WETA
not channel: 31 WWPB
channel: 32 WHUT
not channel: 45 WBFF
channel: 50 WBDC
not channel: 53 WNVT
not channel: 54 WNUV
channel: 56 WNVC
not channel: 60 WWPX
not channel: 62 WFPT
not channel: 64 WZDCLP
channel: 66 WPXW
not channel: 67 WMPB
not channel: 68 WJAL

and i'm using tv_grab_na



And a side note:
I was able to use the webadmin program guide using these instructions (http://www.snapstream.com/Community/Articles/progguide/) to enable the listings in webadmin, but I still can not access it within PVS.

[Ralphus]

Does anyone have a slick batch file / Windows Task Scheduler
entries for automating all this stuff?

Thanks,

Ralph

[ctebo]

Given that you're able to see the xmltv data from the web app, the importer populated the snapstream database correctly.

So this then suggests that something is going on under the hood with snapstream....

Some things to check. Is snapstream using the "default lineup" as its channel lineup? The importer assumes that this is the channel lineup that you are using. The easiest way to guarantee this to be the case is to stop snapstream (ie. exit fullscreen mode and kill the tray icon app). Then delete the C:\Documents and Settings\All Users\Application Data\SnapStream\Personal Video Station\LineupCollection.xml file. And finally, run the Snapstream Setup wizard to re-scan your channels and build the default lineup.

As well, can you double check that snapstream.net automatic updates have been disabled? (web-admin -> configure -> snapstream.net settings -> allow automatic updates -> disabled)

[ctebo]

Automating your updates can be done quick and dirty as follows:

Create a .bat file that first executes your xmltv grab, and then starts the PVS XML Importer. It'll be something like:

"C:\Program Files\XmlTv\xmltv.exe tv_grab_XXX --days Y > outputfile.xml"
"C:\Program Files\PVS XML Import\ImportXMLTV.exe"

Then to have this batch file run automatically, create a scheduled task for it. Do this by going to Start Menu -> All Programs -> Accessories -> System Tools -> Scheduled Tasks. From there you can create a task that executes the .bat file for you every few days at 4am in the morning, or whatever works for you.

Some things lacking in this approach are: if the xmltv grab fails, the importer will dump your exisiting data. Ideally the batch file should identify the error and retry, or abort processing. The next release of the importer will fix the code so that the importer won't dump your existing guide data until the xmltv file has been confirmed to be valid.

Chris

[Ralphus]

Thanks.. that's what I expected. I was wondering if anyone
had an elegant solution to the error situation. Too bad we
aren't programming in in something a bit more expressive than batch files.

Cheers,

Ralphus

[ctebo]

I'm working on extending the importer's functionality to execute the xmltv grab internally, and then trigger the import into pvs. This should be available in a few weeks. Until then, the rough batch approach will have to do.

[heath]

I get an error when running the config program.

I installed using the Windows installer and every time I run the config program, I get Error: 6 Overflow.

Any hints?

[ctebo]

I get an error when running the config program.

I installed using the Windows installer and every time I run the config program, I get Error: 6 Overflow.

Any hints?

First I've heard of this type of error. Does this happen immediately after the config program starts? Does the config program's main window get displayed before the error. Or is this happening after you click on one of the buttons.

What OS are you running this on?

[heath]

Originally posted by ctebo


First I've heard of this type of error. Does this happen immediately after the config program starts? Does the config program's main window get displayed before the error. Or is this happening after you click on one of the buttons.

What OS are you running this on?

Running on XP Professional.

Happens as soon as I start the Config program - no main window, just this error - see attached.

[ctebo]

Thanks heath,

Given that no main window appears before you get the error, only a small portion of code can be involved. I'll dig into the code when I get home tonight to see if I can't get things worked out.

A couple other things that you could try or check...

1) In the C:/Program Files/PVS XML Import directory, do you see a file called config.ini. If so, could you post its contents here?

2) If the config.ini file exists, you could try renaming it so that a new one is created. It might be that there is something in the file that is breaking the config app.

[heath]

Originally posted by ctebo
Thanks heath,

Given that no main window appears before you get the error, only a small portion of code can be involved. I'll dig into the code when I get home tonight to see if I can't get things worked out.

A couple other things that you could try or check...

1) In the C:/Program Files/PVS XML Import directory, do you see a file called config.ini. If so, could you post its contents here?

2) If the config.ini file exists, you could try renaming it so that a new one is created. It might be that there is something in the file that is breaking the config app.

Interesting, there is no config.ini -- maybe it cannot create one?

[heath]

Originally posted by heath


Interesting, there is no config.ini -- maybe it cannot create one?

Just created an empty file called config.ini and restarted the config program. It now works fine.

I think there was some kind of problem with the program creating a new config.ini file for me.

[EDIT] sorry to keep replying in this thread... but my config file is always empty....

What are the options that should be in it, so it knows where to find my database, etc?

[ctebo]

Copy the contents below into the config.ini file in the PVS XML Import directory. You should change the xmltvfile entry to point to your xmltv output file, and the PVSPATH value should be correct. The timezone value should be your UTC time offset multiplied by -1. That is, if your timezone is UTC - 5, then the TIMEZONE value should be 5. If your timezone is UTC + 5 then the TIMEZONE value should be -5.

Once you have saved the file, try running the configuration tool again. You'll need to edit your channel lineup before trying to run an actual import.

Anyway, it sounds like there is a permission issue reading or writing to a file in that directory. We'll figure it out. Can you try logging in as admin and then running the config tool?


[SETTINGS]
TIMEZONE=5
TIMEZONEMINUTES=0
XMLTVFILE=C:\Documents and Settings\ctebo\My Documents\My Code\cvs.sourceforge.net\pvsxmlimport\v1.0\TestEnv \rogers-10days.xml
PVSPATH=c:\documents and settings\all users\application data\snapstream\personal video station\ss_pvs_db.mdb
SILENT=False
AUTOCLOSE=False
SPORT=Sport
MOVIE=Movie

[heath]

The user account I was using before had Administrator rights.
But, I logged in as Administrator, deleted the config.ini and got the overflow error again.

Again, I created an empty config.ini, and it allows the program to run.

I'll set up the ini file manually for now.

[SubZone4]

Hi Guys

I'm trying to get the Australian XMLTV listing to work with the PVS importer, and I'm not having much luck.

Details:

PVS importer version: 1.14
http://prdownloads.sourceforge.net/pvsxmlimport/xmltvimport-installer-v1.14.msi?download

XMLTV version: 0.5.24
http://membled.com/work/apps/xmltv/
(running on RedHat 9)

Australian XMLTV grabber: tv_grab_au-0.2.tar.gz
http://www.onlinetractorparts.com.au/rohbags/xmltvau/

Snapstream version: 3.3
(running on WinXP SP1)

I'm attaching the XMLTV file that that Australian grabber generates (ebroadcast.xml).

When I run the Snapstream XMLTV importer, I set up the configuration. But when I click on the "Edit Channel Lineup" button, I get the error "Run-time error '424': Object required".

If I try to run an import, I get the following output:
[4/12/2003 10:30:24 AM] ---------------------------------------------------------
[4/12/2003 10:30:24 AM] XMLTV to PVS3.0 EPG importer.
[4/12/2003 10:30:24 AM] ---------------------------------------------------------
[4/12/2003 10:30:24 AM] Starting run at 4/12/2003 10:30:24 AM
[4/12/2003 10:30:24 AM] XML filename is: C:\xmltv\ebroadcast.xml
[4/12/2003 10:30:24 AM] EPG database file is: C:\Documents and Settings\All Users\Application Data\SnapStream\Personal Video Station\SS_PVS_DB.mdb
[4/12/2003 10:30:24 AM] Video Library file is: C:\DOCUMENTS AND SETTINGS\ALL USERS\APPLICATION DATA\SNAPSTREAM\PERSONAL VIDEO STATION\SlaveStoreCollection.xml
[4/12/2003 10:30:24 AM] Recording Manager file is: C:\DOCUMENTS AND SETTINGS\ALL USERS\APPLICATION DATA\SNAPSTREAM\PERSONAL VIDEO STATION\JobCollection.xml
[4/12/2003 10:30:24 AM] Channel Lineup file is: C:\DOCUMENTS AND SETTINGS\ALL USERS\APPLICATION DATA\SNAPSTREAM\PERSONAL VIDEO STATION\LineupCollection.xml
[4/12/2003 10:30:24 AM] Time zone adjustment is: -11:00
[4/12/2003 10:30:24 AM] EPG database identified as version 33
[4/12/2003 10:30:24 AM] Clearing old data from EPG
[4/12/2003 10:30:24 AM] Retrieving EPGIDs for shows in your Video Library
[4/12/2003 10:30:24 AM] Found 4 EPGIDS in your Video Library
[4/12/2003 10:30:24 AM] Validating EPGIDs for Recording Manager Jobs
[4/12/2003 10:30:24 AM] Checking for new channels in XML file
[4/12/2003 10:30:25 AM] The XMLTV data file is invalid.
[4/12/2003 10:30:25 AM] Processing Aborted.

I'm also attaching the XMLTV.DTD file I'm using - which is the standard file used.

By the way, I've gone through and used the win32 binary of xmltv to pull listings from the UK, just to ensure that the importer is working correctly, and all is fine.

Now if I could only get the Australian listings going....

Any help would be greatly appreciated.

Regards,
Ron.

[ctebo]

It seems that the australian grabber that you're using is producing xml files that don't satisfy the xmltv.dtd.

The problems in the sample that you provided are:

1) <date lang="..."> : The xmltv.dtd does not support a lang attribute on these elements.
2) <video><color>no</color></video> : The xmltv.dtd supports <colour> tags within the video tag, not <color>.

To manually correct the file you sent me, I used notepad to do the following search and replace functions:
1) Replace <date lang="en"> with <date>
2) Replace <color> with <colour>
3) Replace </color> with </colour>

At that point the channel scan worked using the config tool.

But the bad news is that the program data that you have does not include any end-times for your shows, so there isn't enough data in the file as it stands to successfully run an import. The end-time is recorded as the "stop" attribute on <programme> tags.

The good news is that you can run the tv_sort routine that comes with xmltv to build endtimes for your shows based on the sequence of start-times on a channel. The windows command to do this would be:

xmltv tv_sort ebroadcast.xml > ebroadcast-sorted.xml

If you then run your import against ebroadcast-sorted.xml, your guide data should be populated.

As well, I'll tidy up the error handling so that you would get a more meaningful error message against these data files.

[ctebo]

Did things work when you manually created the ini file?

[SubZone4]

Excellent ! That worked.

After doing the search/replace suggested, and then doing:
xmltv tv_sort < ebroadcast.xlm > au.xml

...this then succeeded in doing an import, including end times being correct.

Now to automate all the steps (tv_grab_au in Linux, spitting it out to a WinXP via a samba client, doing the search/replace, doing a tv_sort, and then the import).

Shouldn't be too bad.

Thanks very much for your help......I'll be eagerly awaiting your instructions on how to get XMLTV working with ActiveState Perl. That way I might be able to get away with not needing to run a Linux virtual machine just to do the Australian grab.

Thanks again,
SubZone4.

[SubZone4]

Just thought that I would share the perl script I created with you guys.
This script will take a tv_grab_au
http://www.onlinetractorparts.com.au/rohbags/xmltvau/
and then run the necessary conversions to get this info correctly into SnapStream via the PVS importer.
Big thanks to Ctebo for helping me out with finding the required conversions.

Here's the perl script:


#First we convert the tv_grab_au output to the correct format

$inputfile = "au.xml"; #this is the initial tv_grab_au file
$outputfile = "au-converted.xml";
$sortedfile = "au-sorted.xml"; #this is the file that PVS importer should import

print "\nConverting the original output of tv_grab_au file to standard format...\n";
open (FILEIN,$inputfile) || die "Cannot read from $inputfile";
open (FILEOUT, ">$outputfile") || die "Cannot write to $outputfile\n";
while (<FILEIN>) {
s/<date lang="en">/<date>/gi;
s/<color>/<colour>/gi;
s/<\/color>/<\/colour>/gi;
print FILEOUT $_;
}
close FILEOUT,FILEIN;


#Then we run a sort, in order to deal with the fact that there are no specified ending times for the programs
print "Sorting the XML file...\n";
`xmltv tv_sort < $outputfile > $sortedfile`;


#Lastly, we run the PVS importer
print "\nRunning the SnapStream PVS importer...\n";
`C:/Program Files/PVS XML Import/ImportXMLTV.exe`;

print "\nDone.\n\n";




Regards,
SubZone4

[heath]

Originally posted by ctebo
Did things work when you manually created the ini file?

Once I manually created a config.ini file, I find that the config program does write to it when I change the settings.

After configuring my channels, the database is correctly updated, but I notice that my channel lineup is not configured in SS. It should be putting all of the channel names and aliases in the channel lineup shouldn't it?

[ctebo]

After configuring my channels, the database is correctly updated, but I notice that my channel lineup is not configured in SS.

Something strange is going on there. When you edit your channels in the config app, it should write your call signs to the LineupCollection.xml file in the Snapstream Application Data directory.

I suspect this is somehow related to file permissions and privileges. I will play around with permissions on my setup to see if I can find a way to reproduce the problems you've been seeing.

[heath]

Originally posted by ctebo


Something strange is going on there. When you edit your channels in the config app, it should write your call signs to the LineupCollection.xml file in the Snapstream Application Data directory.

I suspect this is somehow related to file permissions and privileges. I will play around with permissions on my setup to see if I can find a way to reproduce the problems you've been seeing.

Note: I am able to edit the channel listings (using the same login id) with the PVS web configurator, but not with the XMLTV import configuration program. So I doubt it is user permissions, but maybe some API that was chosen or incorrectly used?

[ctebo]

I still suspect that the initial problem was a file permissions issue. As snapstream doesn't provide APIs, the importer works by operating against snapstream's data files. I know that might sound scary, but the snapstream folks have been good about giving us a heads up as releases introduce changes that would break our integration.

As well, something unique is going on in your install as this functionality has been working just fine for several hundred users.

The one possibility here is that you are using a channel lineup other than the "default lineup". Is that the case? The importer assumes that the "default lineup" is the one in use.

[heath]

I am using "default lineup", although I have recently added a second lineup for my alternate input...

I did the tests before I had added a second input.

[SubZone4]

Hi Ctebo

Could you please let us know when your importer program supports the latest 3.4 beta (build 835) that just came out. The importer seems to successfully complete the import, but nothing appears in the program guide.

Thanks,
SubZone4

[ctebo]

Check the beta forum for a thread covering the availability of an update.

[miniGranis]

I have been using the Importer with data from tv_grab_sn getting Swedish channel data for a long time and everything was working fine.

Due to changes in the feed for tv_grab_sn we can no longer use it. People have however come up with a replacement called tv_grab_se that works fine for grabbing data.

The import to SnapStream works for about 95% but sometimes I get UNAVAILABLE in the EPG eventough data exists in the XML file.

Example from XML file:
source-data-url="http://www.tvprogrammet.nu/" generator-info-name="XMLTV" generator-info-url="http://membled.com/work/apps/xmltv/">
<channel id="Kanal 5.txt.tvprogrammet.nu">
<display-name>Kanal 5</display-name>
</channel>

### CUT ###

<programme start="20031213200000 +0100" stop="20031213210000 +0100" showview="219435" channel="Kanal 5.txt.tvprogrammet.nu">
<title lang="sv">High School reunion</title>
<desc lang="sv">Amerikansk realityserie från 2003. Del 6 av 6. Det är dags för studentbal. Därefter är den tumultartade återföreningen över. Även 14/12.</desc>
</programme>
<programme start="20031213210000 +0100" stop="20031213002000 +0100" showview="36082690" channel="Kanal 5.txt.tvprogrammet.nu">
<title lang="sv">The postman - budbäraren</title>
<desc lang="sv">Amerikanskt äventyr från 1997. USA år 2013. Landet är splittrat och kommunikationer har slutat att fungera. Till en liten by kommer en mystisk vandrare som inger hopp. Med rättvisa som ledstjärna leder han ett uppror mot den fascistiske ledaren Bethlehem. I rollerna: Kevin Costner, Will Patton, Larenz Tate. Regi: Kevin Costner.</desc>
</programme>
<programme start="20031214004500 +0100" stop="20031214010000 +0100" showview="8175115" channel="Kanal 5.txt.tvprogrammet.nu">
<title lang="sv">Tomtar på loftet</title>
<desc lang="sv">Repris från i dag.</desc>
</programme>


In the above case the first show "High School reunion" gets imported.
The show "The postman - budbäraren" doesnt show up in EPG
The show "Tomtar på loftet" shows up in EPG

Here is a screenshot using the html EPG browser:

http://minigranis.no-ip.com/tmp/epg_guide.jpg

Please note that I have included two screens of EPG data below each other to show that "Tomtar på loftet" does show up after the UNAVAILABLE /UNEXISTING "The postman - budbäraren" show.

Can anyone give me a hint on whats wrong ??

Brg
Emil
Ps
Sorry for the long post
Ds.

[ctebo]

In looking at the sample output from called tv_grab_se that you posted, the problem isn't with snapstream or the importer. Its actually a problem with the data that tv_grab_se is producing.


<programme start="20031213210000 +0100" stop="20031213002000 +0100" showview="36082690" channel="Kanal 5.txt.tvprogrammet.nu">
<title lang="sv">The postman - budbäraren</title>
<desc lang="sv">Amerikanskt äventyr från 1997. USA år 2013. Landet är splittrat och kommunikationer har slutat att fungera. Till en liten by kommer en mystisk vandrare som inger hopp. Med rättvisa som ledstjärna leder han ett uppror mot den fascistiske ledaren Bethlehem. I rollerna: Kevin Costner, Will Patton, Larenz Tate. Regi: Kevin Costner.</desc>
</programme>


This show was not successfully imported into the epg. Look at the start and stop times that tv_grab_se produced:

start="20031213210000"
That is, December 13 2003, 9:00pm

stop="20031213002000"
That is, December 13 2003, 12:20am

This data doesn't make any sense - the stop time is about 21 hours before the start time. The importer does some validation on the start and stop time data and throws away shows that do not provide a stop time, or have a stop time before the start time.

I'd suggest that you chat with the folks maintaining your new grabber to see if they can't fix their data collection process.

[miniGranis]

Originally posted by ctebo
In looking at the sample output from called tv_grab_se that you posted, the problem isn't with snapstream or the importer. Its actually a problem with the data that tv_grab_se is producing.
time.

I'd suggest that you chat with the folks maintaining your new grabber to see if they can't fix their data collection process.

Thanks for you very quick reply !!
You are indeed right - I totaly missed this.
There is a problem with the grabber for shows spanning over midnight.
I will contact the developers.

Thanks again !!

/Emil

[Camian]

:) Great!

Emil, can you (or someone) tell me where I can find tv_grab_se?

/ Mike

[miniGranis]

Originally posted by Camian
:) Great!

Emil, can you (or someone) tell me where I can find tv_grab_se?

/ Mike

Here you go:

http://article.gmane.org/gmane.comp.tv.xmltv.devel/2476

see also:
http://article.gmane.org/gmane.comp.tv.xmltv.devel/2502

I have made some "dirty" changes to fix the midnight issue for the time beeing:

# Example: 06:25-06:25|Barnprogram||4037809||

my @lines = split(/\n/, $data);
my @programs;
my $today = UnixDate($date, '%Y%m%d');
my $today_start;
my $today_stop;

... SNIP...

if ($showview) {
$program{showview} = $showview;
}

if ( $raw_start gt $raw_stop ) {
print STDERR "Correcting stopdate for show: $ch_xmltv_id: $title starting at $raw_start ending $raw_stop\n";
#print "RAW_START: $raw_start RAW_STOP $raw_stop\n";
$today_start = $today;
$today_stop = $today+1;
#print "today_start: $today_start, today_stop: $today_stop \n";
$program{start} = utc_offset("$today_start $raw_start", $BASE_TZ);
$program{stop} = utc_offset("$today_stop $raw_stop", $BASE_TZ);

}
else {
$program{start} = utc_offset("$today $raw_start", $BASE_TZ);
$program{stop} = utc_offset("$today $raw_stop", $BASE_TZ);
}

#print "title: $title, $program{start} $program{stop}\n";
$w->write_programme(\%program);
}

return 1;
}


I hope you will understand what I did and what to edit.
I am using ActivePerl under w2k to run the script.

Brg
Emil

[elneckio]

I have a small suggestion for the importer interface that might save new users some setup time.

When editing channel lineup within the importer it checks for conflicts between the numbers you have typed in....

If there is a conflict it tells you and won't save the changes until you fix it. It would be nice at this point if it told you the channel number that is duplicated, and also as a further help which pages the number appears on or which channel name.

Just an idea :)

[ctebo]

Good point elneckio,

When I first introduced the new channel lineup editor I figured some help along the lines you describe was needed, but then figured that since using the original channel editor was like pulling teeth, I could get away without having those niceties in the editor at that time.

I suppose the time has come to tidy some of that up... I'll make some progress on that for the next release.

Chris

[elneckio]

Thanks Chris :D

[bryan_chow]

I had XMLTV working perfectly. Then I built a new server dedicated for SnapStream, and copied all my XMLTV files over to the new server but it no longer works.

XMLTV runs fine and processes the xml file, but the programmes do not show up in the EPG.

When I try to access the ASP (web-based) guide, I receive an error saying the database password is incorrect. Did the database password change from two months ago??

Thanks,
Bryan

[ctebo]

Some more detail on you config would be helpful in tracking your problem down...

What version of SnapStream? What version of the XMLTV importer?

Can you post the contents (for a recent run) of your XMLTVImporter.log This file can be found in C:/Program Files/PVS XML Import/ (assuming that you're using an importer version > 1.14).


Chris

[ctebo]

For information on XMLTV imports with BTV 3.4, please follow this thread:

http://discuss2.snapstream.com/vb/showthread.php?s=&threadid=11336

[TFLeonard]

I was trying to follow the instructions on importing XLMTV to Beyond TV and could not get the http://www.snapstream.com/download/ss_asp_epg.zip file since the link is broken (file not found).

Is there another source for this or could someone e-mail it to me at:

tfl@coralwave.com

Thanks

[elneckio]

I downloaded and installed the new 3.4 - 3.41 just to stay up to date. And it trashed my channel lists again - great !. And when run it says it is 3.4 build 899 :(

Anyway just wondering if you had put a conflicts message box in yet ctebo ?

I don't fancy debugging the conflicts & typos by hand again ;)

[ctebo]

1.17 of the importer will make it easier to manage your channel lineups. It will do so in two ways:

1) The "edit channel lineup" gui will move all conflicting/blank channels to the front of the channel list. This should make it easier to figure out and get through the process of dealing with conflicts.

2) Since it seems that BTV upgrades will always trash your channel lineup, the importer will maintain a copy of your most recent channel lineup after an edit. When you run the configuration tool and it realizes that your channel lineup has been trashed, it will prompt you to restore your most recent channel lineup.

These changes should be available by the end of this week. Sorry it won't be sooner, but with the holidays and all...

Chris

[elneckio]

This sounds real nice :)

Out of interest what is the filename of the file it trashes ? - It's possible I may still have a backup of it.

Can you just overwrite it with the backup version ?

[ctebo]

The file that contains your lineup is called "LineupCollection.xml"

However, backing it up isn't enough with 3.4. BTV expects the contents of the epg database (which contains channel info) to be in sync with the xml file.

[ctebo]

A new build of the importer is now available that corrects the "Type mismatch 13" error some of you were seeing during imports. You can donwload it from here: http://pvsxmlimport.sourceforge.net

This was caused by incorrect handling of shows spanning year boundaries with negative timezone offsets.

Post here if you still have any remaining issues after upgrading.

Elneckio, the channel lineup changes are not visible in this new release - this one is pushed out to correct the issues around year boundaries. They will now show up in v1.18. Next weekend.

Chris

[elneckio]

ctebo - no rush for that feature, I added a messagebox to the sourcecode and made a quick build here..

It displays which pages the conflicts are on and the details of the two items. Not as clever as your plan but it worked :)

I can email you the change if it would be of any use to you ?

[mangiov]

hi ctebo,
I sucessfully loaded already 2 batches of listings with the latest version of the importer. An odd thing has happened both times. All the series I am recording do find any match on the freshly loaded EPG; fo example, I record Pokemon for my son that is out to school. The show is on the same channel everyday at the same hour. I set up a scheduled recording for all episodes and everything has always worked fine. Starting from this latest version I have to delete the old schedule as it doesn't "see" the show anymore and I have to setup a new schedule.

Hope I have explained it well :).

Thanks,
Vinny

[rmanke]

Whenever I run the Config app, I get the following error:

---------------------------
Runtime Error
---------------------------
configMain.Form_Load

Source: ConfigData.GenerateConfig : xmlconfig

Number: 6

Description: Overflow

Application will exit.

---------------------------
OK
---------------------------


I have followed the steps outlined above to get the XML Importer to work with the latest version of BeyondTV.


Any ideas?

[beles]

Hi

as the german part doesn't work with the recommended utitly I use Nextview EPG to extract the data.

Problem that some shows doesn't get importet into snapstream. I have attached an example file. "Jamie's Kitchen" doesn't import until I have removed the:

<category lang="en">social/political/economics</category>
<category lang="de">Soziales/Politik/Wirtschaft</category>
<category lang="fr">Socio/Économie/Politique</category>

tags. It looks like it is related to the slash or something. but other categories with a slash do work.

but otherwise the software is great. thanks a lot for providing it.

cheers
Petr

[ctebo]

Based on the error info you provided, it seems something is going wrong during the calculation of your timezone offset. To help me try to reproduce this, can you provide:

1) The timezone setting that your pc is set to?
2) What operating system are you running?
3) At rougly what time were you trying to run the configuration?

Thanks,
Chris

[ctebo]

Glad to hear you're having success populating guide data using Nextview EPG to extract the data.

It seems that the issue you were running into is related to those category fields, in that BTV restricts the contents of its genre field (which is where the category information is placed) to 150 characters. The importer at present does not restrict the length of that field, and hence the issue you have been seeing.

I've corrected the code to truncate the genre field at 150 characters to provide the quick solution. This fix will be included in the release of 1.18 of the importer. Timeframe is likely a couple weeks.

Chris

[ctebo]

Are you running BTV 3.4 when you experience the problem with recording jobs not working after an import?

Chris

[mangiov]

ctebo,

yes, I am running the latest PVS build.
I probably should have posted my mssage on the other thread :o

Vinny

[beles]

Hi Chris

1) The timezone setting that your pc is set to?
My PC is running in Zurich which is European Time = +1:00. I have set the Nextview software to this timezone and the file looks right to me when I compare it to real showtimes. I have set the importer to +0:00 as it did import the times wrong when I did set it to +1:00 (all shows where 1 hour to late in the showview EPG).

2) What operating system are you running?
Windows XP Professional

3) At rougly what time were you trying to run the configuration?
Somewhere in the evening around 11pm but tried also in the morning 10am. The problem was reproducable. Removing the mentioned tags solved the problem. Adding them again made the show dissapear again.

The most shows where imported correctly just realy a few didn't show up in the EPG. If the informations doesn't help I can try next week-end to figure out if there is any relation between the shows which weren't imported.

Thanks you for your help
cheers
Petr

[beles]

Hi

sorry. forget my post. mixed up things

[beles]

Hi Chris

got now the right message :)

>It seems that the issue you were running into is related to those category fields, in that BTV restricts the contents of its genre field (which is where the category information is placed) to 150 characters. The importer at present does not restrict the length of that field, and hence the issue you have been seeing.

You are absolutly right, after extending the genre related database fields to 255 characters lenght all shows imported without problems. Maybe the snapstream developers could check if it would be possible to change this in the next release if it doesn't cause any other problems.

>I've corrected the code to truncate the genre field at 150 characters to provide the quick solution. This fix will be included in the release of 1.18 of the importer. Timeframe is likely a couple weeks.
Thanks a lot

cheers
Petr

[srigby]

Originally posted by mrphil
What if I change the code so it also allows you specify just a directory, and it will import all files it finds in that directory? W=Is this what your looking for?

Im thinking also would it be a good idea to delete each file after it has been processed, so you don't import it again next time things run.

Hiya Ctebo,

Appreciate the excellent work you're doing.

Did you ever get around to adding the above feature to ImportXMLTV? My XMLTV grabber writes individual files when it grabs multiple days (eg: 20040101.xml, 20040201.xml).

Was the code modified to allow importation of multiple files in a directory, rather than just one file?

Thanks again...

[mickwall]

Anyone else in the UK having trouble with TV listings. This is on the ananova site

Get TV listings on the move
Sorry, but Ananova is no longer able to bring you TV listings.
However, you can get a complete seven-day listings guide for terrestrial and satellite TV provided by Radio Times on your Orange phone.
With Orange World you can also keep up with the latest TV news and gossip, buy videos and DVDs of top TV shows and download the ringtone of your favourite theme tune.


I hope this is not the end of TV listings for us in the uk!

[elneckio]

Thought of this when I found out :
"I felt a great disturbance in the Force...as if millions of voices suddenly cried out in terror and were suddenly silenced. I fear something terrible has happened."

Seriously though, I just read on myhtpc that someone is working on a replacement, hope he can find a source ok.

http://myhtpc.net/forum/viewtopic.php?t=5634&highlight=ananova

Maybe this will push priority up for official snapstream uk listings support (maybe :) ).

[peter]

Maybe somebody from SS could comment on the likelyhood of proper UK listings available from SS.net.

It was mentioned (a long time ago) that UK & Canada would probably be next after the US, now that Canada has its listings how about the UK listings?