PDA

View Full Version : Is it possible to limit number of concurrent post-processing tasks?


cmcquistion
07-04-2006, 09:30 AM
I just upgraded my BTV server from a single-core AMD CPU to a dual-core AMD CPU, because I was having stuttering problems, from BTV Link, when the BTV server was running a showsqueeze job. I thought that a dual core system would solve this, by leaving enough CPU power for the Link connection to not struggle.

I was wrong.

Now, because I have a dual core system, BTV runs two post processing tasks at a time. This means that I get more tasks done quicker, but it also means that the system is sometimes running two showsqueeze tasks at a time, which creates problems for my connection from the Link system, again.

Is there a way to modify one of the XML files or something, so that BTV only runs one task at a time? Ideally, I would like to make it so that it can run two tasks at a time, during the night, but limit it to one tasks at a time, during the day. This may be asking too much, but if it is a simple variable in an XML file, I might be able to use Scheduled Tasks to overwrite the file a couple times a day, assuming that BTV reads this file when it has a task to start and doesn't just read it, when BTV starts up.

Any ideas where/how I can change this variable?

bnflaherty
07-04-2006, 09:33 AM
That's odd...

See my sig for my setup -- I don't have Any stuttering problems whatsoever...

My BTV machine is either recording or squeezing... we watch live tv while squeezes occur -- with no issues.

What quality are you using?

cmcquistion
07-04-2006, 10:32 AM
I'm using a custom DivX profile to showsqueeze all programs (HD or SD) to 720x480, two pass encoding, 2200 kbps video, 192 kbps audio, resize style set to Keep Aspect Ratio, Resize Algorithm set to Bilinear, Bidirectional frames set to 2, Noise Filtering Off, Performance Mode Standard, Psychovisual Mode Off. This results in a file size of about 1 GB/hr.

My server also has 4 capture devices, which can capture up to 6 simultaneous programs ( two of them are dual tuners), though I am rarely capturing more than 3, simultaneously.

I also have a Gigabit ethernet connection between server and Link. I have QOS enabled on all network cards and 9K jumbo frames turned on.

nanook105
07-04-2006, 10:47 AM
Does Live TV stutter on the BTV server while running these processes ?

Does the link stutter while watching a recording or just while watching live TV ?

I've never noticed stuttering on my links but I'm not sure that I typically use them while showsqueeze jobs are running. I'll play around to see if I get the same results.

cmcquistion
07-04-2006, 11:12 AM
I'm not sure whether live or recorded TV stutters on the main server, during showsqueeze. I never actually watch TV on the server.

During showsqueeze jobs (multiple), I see very jerky playback on the Link machine, when playing back pre-recorded shows (also see incredibly sluggish performance in the menus.) I'm not sure about Live TV, since I rarely watch anything, Live, but I'm fairly certain that I would have the same problems, there.

bnflaherty
07-04-2006, 11:18 AM
for the heck of it -- have you monitored the network usage?

cmcquistion
07-04-2006, 11:57 AM
I'm trying to recreate the problem, now. I'm showsqueezing two shows.

The menus are a little sluggish on the server. They are much more sluggish on the Link machine. I've been watching the network usage on both machines to see if that seems to a variable. It never goes over 1%, in the menus. While I'm watching a show, it usually sticks around 1-2%, going up to 15%, if I do fast-forwarding or rewinding.

Of course, since I'm trying to recreate the problem, I'm having a hard time getting it to recreate the problem.

*EDIT* The problem (video playback stops completely or becomes really jerky) happens sometimes, but not all the time. The only common denominator that I've seen, so far, is that usually the server is doing two showsqueeze jobs, when this happens.

karhill
07-04-2006, 07:12 PM
perfmon.exe is your friend in trying to get a handle on some of these performance problems. With a dual core machine running multiple squeezes and a simultaneous playback, you might be stressing your disk subsystem. Can you arrange it such that the playback is from a different physical drive than the squeezes? See if that makes a difference. Also watch disk performance in perfmon.

cmcquistion
07-04-2006, 10:55 PM
Generally, I keep the Performance monitoring service disabled, but I will enable it and start checking that.

I don't think that disk performance should be my problem. I've got two, brand-new 500 GB WD SATA drives with 16 MB cache, in a Dynamic Volume (spanned). These are pretty high performance drives, so I would expect them to be able to write and read, simultaneously, pretty well, when dealing with such low sustained bandwidth as these recordings.

Rich A
07-05-2006, 11:24 AM
Are those two spanned drives the only drive? Do you have a separate drive for your OS ? If not, I would get a small drive just for the OS and Windows swap file. I would turn off indexing on the storage drives and make sure the windows swap file is on the OS drive only.

I have a RAID 0 external drive array. Plus a 120 GB main drive. I often watch a recorded digital HD, while recording a 2nd HD show, and two analog shows.

And I can access the server with TWO link machines while all this is going on without any performance hits.

Two showsqueeze jobs at the same time will have more of an impact on the drive I/O than the CPU when you have a dual CPU system. While each CPU can run each own showsqueeze process, you are still using only one drive. (the big spanned one)

Each squeeze job has to both read and write files. So now you are doing dual reading and dual writing to one drive. Yes a stripped raid type configuration will help increase the speed that the drive writes and reads, but the performance gain won't be as great if you have two applications banging on it. Add to that the over-head from the OS requirements and it could be problematical when you add still another read requirement (ie a link box playing back a file)

The first thing I would do is to insure Windows Indexing service for the storage drive is turned off. If you are continually writing to the drive (like building a new showsqueezed file) the indexing is going to be working overtime.

Today's drives are of course very fast. I would think your system could handle it. I would go the separate drive route only as a last resort.

cmcquistion
07-05-2006, 01:36 PM
I have a separate, smaller drive for Windows and the swap file. I have indexing turned off on all the drives.

Rich A
07-06-2006, 01:37 PM
Well then you have got some weird thing going on .. and since it is software version specific, I'd have to say it's only going to be fixed by some bug fix.

The question remains .. what "thing" in the latest version of BTV was changed that would affect you?

Oh .. going back to a long time ago .. I had a similar problem. Stuttering that suddenly happened with a new software release. I had found that somehow the Quartz.dll was over-written with a newer version during the BTV update. I put back the older orignal Quartz.dll and my stuttering disappeared. That was at least 3 or 4 revisions ago. I've not had the problem since. Search these forums for "Quartz.dll". Although I doubt it will help, it does show that sometimes something is changed by the install which can cause problems.

I'm guessing that the latest Quartz.dll as supplied by BTV wasn't compatible with my system, most likely because it had been a long time since I had done any Windows update. Soon after changing back to the older DLL I decided to catch up on my Windows updates. I think those updates may have made my Windows environment more proper to work with the newer Quartz.dll. I only say that because I'm sure during the subsequent 3 or 4 BTV releases that DLL was once again updated .. only now it is working properly.

You might check for any new Windows Updates .. and install them and then re-install BTV.

That scenario was (I'm pretty sure) the only time I had a new BTV release cause a problem in the last couple years. (lucky I guess) :wink:

cmcquistion
07-06-2006, 01:58 PM
The one thing that I can put my finger on is the Streamsnip. I have this turned on and if it is running on a recording that I'm watching, things get pretty hairy. It will prematurely end the program, but if I exit and then restart watching the program, it works fine. I believe that this has to do with it cutting the program down to a smaller size and then the program's timeline thinks the recording is over, because it is basing playback on the previous filesize and the new filesize (I'm probably not explaining this very well.)

I wish that I had some control over the various post processing tasks and when they run; Streamsnip, in particular. I love the feature, but it annoys the heck out of me and my wife, when it runs on a recording, while we are watching it.

This, however, is not the common thread. I'm having difficulty sometimes when Streamsnip is running on a completely different recording, or when I have multiple jobs going, usually when one or more of them are showsqueeze jobs.

I'll keep watching. The difficult part is that this isn't easily reproducable. Sometimes the issue will happen for several hours, but then it might not happen again for two days.

ZetaVu
10-13-2006, 02:03 PM
Ok, back to the original question, is there any way to have only one showsqueeze at a time on dual core systems? I just noticed this on my system, (since upgrading) I'd really rather do one showsqueeze at a time.

btvfreak
10-13-2006, 02:31 PM
You can always just use the "Time Constaint" and have the showsqeeze jobs only run late night when your BTV usage is low to none.

There is something odd going on with your setup though. I have a 3800+ AMD X2 and I can have several jobs going on (showsqeeze, comskip) while playing back shows or viewing LiveTV with no problem.

ZetaVu
10-13-2006, 02:33 PM
I'm not have playback issues, I just want to see if I can have one showsqueeze run at a time.

cmcquistion
10-13-2006, 04:13 PM
Snapstream support did not help me with this issue, but I did figure out how to force only a single showsqueeze at a time, using a set of custom batch scripts (one of which uses Comskip to create a smartchapter file.)

See my post in this thread, starting at post 25:

http://forums.snapstream.com/vb/showthread.php?t=31631

Basically, I have Custom Post Processing tasks that looks for evidence of a running Showsqueeze job and if its find it, it waits 60 seconds and then looks again. This repeats until it can't find evidence of a running showsqueeze job. It then exits and the next post processing task (showsqueezing) can get started.

ZetaVu
10-13-2006, 06:00 PM
Nice approach, I'll give it a try as soon as I finish off the new install bugs. Thanks

pwlong
09-28-2007, 11:30 PM
Not sure if it's already been mentioned, but this feature has been implemented in 4.7.1:

"Number of Simultaneous ShowSqueezes
By default Beyond TV will detect the number of processor cores in your system and use that number to determine the number of simultaneous ShowSqueezes it can perform. However, if your computer has performance problems ShowSqueezing more than one file at a time or Beyond TV is not correctly detecting the number of processor cores in your computer, you can manually set the number of simultaneous ShowSqueezes here."

Choices are Auto-Detect, or the option to specify between 1 and 8 concurrent ShowSqueeze jobs.

WebAdmin -> Settings -> ShowSqueeze -> Number of Simultaneous ShowSqueezes

tscales
09-29-2007, 06:10 AM
As important, you can also specify exactly when custom processes run. I was able to eliminate all the extra programs/work I did for recordings. Love it!