Step by Step: Video fixing, Commercial Detection, and Removal.

[cmcquistion]

I've written a series of scripts, over the past several months that do a few things the BTV can't do, out of the box, and I really wanted.

First, I wanted more accurate commercial detection than Smartskip was providing.

Second, I needed a way to clean up some of my video files. I record over the air HD programming, but because I am about 30 miles from the sources of the video, I sometimes get little glitches in the video. This happens on standard definition MPG videos from time to time, also. It is usually minor and doesn't cause a lot of trouble, but it becomes a problem if that video is compressed, because of lost audio or video frames. The resulting compressed video could develop audio sync problems, if the video file isn't cleaned up, first.

Third, I wanted a way to cut commercials out of certain shows, but not all shows, necessarily. Some channels and some shows have very predictable and accurate commercial detection, using Comskip. Some channels, however, and some particular shows, do not have quite enough accuracy for me to be confident in cutting them out.

To solve the first problem, I tested VideoRedo's Ad Detective and Comskip. I spent a good while testing and tuning each one. In the end, I found Comskip to be much more configurable and accurate. I found a batch file, here in the forum, somewhere, that would allow me to run Comskip as a Custom Post Processing task, after recording. I modified this script a bit, to suit my needs.

The Comskip batch file worked well, but it ran into some problems with video files that were a little damaged and needed some fixing up, especially HD recordings. If I ran Comskip as a Custom Post Processing task, this was run before Streamsnip (on HD recordings). Streamsnip would fix up the file a little bit, but this caused the smartchapter file that was generated by Comskip to get a little off, and get worse as you proceeded further through the video. Ideally, it would be nice to be able to run Streamsnip BEFORE all other Custom Post Processing tasks. I put requests for this feature, starting 6 months ago. I was told that it would be considered for future versions, but it has not been yet been delivered. Solving this problem takes us to the solution to the second issue.

The solution to the second issue (fixing up the video) was solved by getting VideoRedo. VideoRedo is not free software, but it is a fantastic piece of software. It can be used for video editing, commercial detection and removal, or just running a "QuickStream Fix" on a video file. Quickstream Fix can take a Transport Stream or MPG file and re-save it, fixing various errors in the file, while doing so. The program is fast, works well, maintains the embedded BTV program information, and doesn't introduce errors or audio synchronization problems. No other program I tried (mostly free software) could deliver all of these features. Each had their quirks and problems. VideoRedo turned out to be worth the cost. I was able to write up a batch script (with help from the VideoRedo forums) that would take my video, output a Quickstream Fixed version of that video, and then replace the original, with this fixed version. I began running this script before the Comskip (commercial detection) script, and it worked great. My videos were smooth and error free, commercial detection was great, and compressed files didn't suffer from audio synchronization problems. This left me wanting more, which is where the third solution comes in.

The solution to the third problem (removing commercials from the video file) was finally achieved once I got the first two steps working well. I modified my comskip.ini file (the settings file for Comskip) to output a VideoRedo project file, in addition to a BTV Smartchapter file. I then wrote a script to pass my videos through VideoRedo, using the VideoRedo project file (generated from Comskip) to output a new version of the video, minus the commercials. I implemented a variable-matching section in this script so it would check to see if the video file name matched certain parameters. If it did, then it got the commercials cut out. If not, the script just exited. I can do pretty simple phrase matching like "*MythBusters*" to pass all episodes of MythBusters through VideoRedo and cut out the commercials. The final video, without commercials, then replaces the original video file.

I spent quite a while tweaking all of these scripts to make them very simple to pass on. There is only one variable that needs to be explicitly set and each script can be used by itself, or in combination with the other scripts. You can even copy and paste all of these into one script (which is what I generally do) and run it as one long post processing chain.

Here's what you will need to set all of this up. First, you will need to go here to download the appropriate files to set up your own Custom Post Processing tasks. Next, create a directory somewhere that you're going to put your Comskip files (I put all my scripts in this directory, also.) On my system, this is "M:\MEDIA\Files_Backup\Comskip". This drive and path is one of the only pieces of information that you will need to explicitly set in the scripts, below (the comskip.bat script, specifically.) Next, download Comskip from here. Unzip these files into the Comskip directory you decided on. I'm currently running version 79_129, so my configuration files will be specific to that version.

Here is the contents of my Comskip.ini file (for version 79_129).

detect_method=111
validate_silence=1
validate_uniform=1
validate_scenechange=1
verbose=0
max_brightness=60
test_brightness=40
max_avg_brightness=25
max_commercialbreak=600
min_commercialbreak=25
max_commercial_size=125
min_commercial_size=4
min_show_segment_length=125
non_uniformity=500
max_volume=500
min_silence=12
ticker_tape=0
logo_at_bottom=0
punish=0
punish_threshold=1.3
punish_modifier=2
intelligent_brightness=0
logo_percentile=0.92
logo_threshold=0.75
punish_no_logo=1
aggressive_logo_rejection=0
connect_blocks_with_logo=1
logo_filter=0
cut_on_ar_change=1
delete_show_after_last_commercial=0
delete_show_before_or_after_current=0
delete_block_after_commercial=0
remove_before=0
remove_after=0
shrink_logo=5
after_logo=0
padding=0
ms_audio_delay=5
volume_slip=20
skip_b_frames=1
max_repair_size=200
disable_heuristics=4
delete_logo_file=0
output_framearray=0
output_videoredo=1
output_womble=0
output_mls=0
output_cuttermaran=0
output_mpeg2schnitt=0
output_mpgtx=0
output_dvrcut=0
output_zoomplayer_chapter=0
output_zoomplayer_cutlist=0
output_edl=0
output_edlx=0
output_vcf=0
output_bsplayer=0
output_btv=1
output_projectx=0
output_avisynth=0
output_vdr=0
output_demux=0
sage_framenumber_bug=0
sage_minute_bug=0
live_tv=0
live_tv_retries=4
dvrms_live_tv_retries=300
standoff=0
cuttermaran_options="cut=\"true\" unattended=\"true\" muxResult=\"false\" snapToCutPoints=\"true\" closeApp=\"true\""
mpeg2schnitt_options="mpeg2schnitt.exe /S /E /R25 /Z %2 %1"
avisynth_options="LoadPlugin(\"MPEG2Dec3.dll\") \nMPEG2Source(\"%s\")\n"
dvrcut_options="dvrcut \"%s.dvr-ms\" \"%s_clean.dvr-ms\" "
windowtitle="Comskip - %s"

This is my customized comskip.ini file for my particular system. Your mileage may vary. One of the nice things about Comskip is that they give you the ability to tweak a lot of settings, here. You can tune this to your particular system, when necessary. Keep in mind that new versions of Comskip add new variables, so if you use a newer version, you may want to use some of my settings as a guide, but not necessarily use them verbatim.

Next, I'll talk about the actual CustomPostProcessing.xml, which is used to define what post processing tasks are run and when/how they are run.

Here is an example of a CustomPostProcessing.xml file, which is located in "C:\Documents and Settings\All Users\Application Data\SnapStream\Beyond TV"

<CustomTasks>
<Task>
<GUID>531E0175-87F4-4d6f-B464-E603BC641DAE</GUID>
<Priority>Deferred</Priority>
<PropertyCollection>
<Property>
<Name>Command</Name>
<Value>M:\MEDIA\Files_Backup\Comskip\batch.bat</Value>
</Property>
</PropertyCollection>
</Task>
</CustomTasks>

In this example, one batch file (batch.bat) is executed for every recording. It is set to Deferred. If you have Time Constraints Disabled on this screen http://localhost:8129/ShowSqueezeConstraints.aspx then these jobs will execute immediately after a show is finished recording. Beyond TV can detect whether you are running a single processor/core computer or a multi processor/core computer. If Beyond TV knows that you have two CPU's or two CPU cores, then it will execute two post processing tasks at a time. If you only have one CPU/core, then it will only execute one post processing task at a time. (The command, in bold above, should be changed to the proper drive, path, and name of your Post Processing script.) In BTV 4.8.1 or later, if you go to http://localhost:8129/SetupShowSqueeze.aspx and change "Number of Simultaneous ShowSqueezes" to 1, then it will only execute on post processing task at a time, which I highly recommend.

On my system, since I have seven tuners (and just added two more that aren't fully integrated, yet) I have lots of recordings starting and stopping at the same time. In noticed jumpy performance in Beyond TV Link, sometimes because several recordings might stop, several more recordings start, and several post processing jobs start, all at the same time (I have a dual-core CPU, so BTV runs two post processing tasks at a time.) To minimize this slight annoyance, I wanted to implement a script that would delay the initial beginning of my post processing tasks. It only needs to delay them for a few seconds so there aren’t as many I/O operations trying to happen at the same time. I wanted to make a script that would determine a random number and then make the system wait for a random number of seconds. I found my answer with a little Googling. Someone posted a semi-random number generator that I then used to create a small delay script that makes the system wait for 1-99 seconds. To actually implement a command to wait or delay, I installed the Windows Server 2003 Resource Kit, which can be freely downloaded here. Once this is installed, do a search on your C: drive for SLEEP.EXE. Copy this file to the C:\Windows directory and now SLEEP can be called from a command line. This very simply makes the batch file sleep (or wait) for the specified number of seconds. This SLEEP command is used in most of my scripts, so it will need to be installed.

The total post processing chain of events is comprised of a Delay script, then a Quickstream Fix script, then a Comskip script, and finally the Commercial Cutting script. Although this may seem like a lot, these processes are actually fairly quick and the total processing time is not unreasonable. If you want to run these jobs during hours that you don't have to watch them popping up on your screen, you can set Time Constraints on this page: http://localhost:8129/ShowSqueezeConstraints.aspx. You can schedule all of this activity during night hours, if you prefer. My home is comprised of a BTV Server that does recordings and post processing, but I watch television from a separate BTV Link machine, so I don't have to see what goes on behind the scenes. For me, running these jobs all day isn't a problem, so I have the Time Constraints disabled.

Here is the contents of the Delay.bat script:

@ECHO OFF
::
::
:: Delay script to wait a random number of seconds (between 1 and 99)
:: (Last Edited 03-22-07)
::
::
FOR /F "tokens=2 delims=." %%A IN ('VER ^| TIME ^| FIND "current"') DO SET RANDOM=%%A
::
ECHO Sleeping for %RANDOM% seconds, so Post Processing jobs don't bump heads...
ECHO .
SLEEP %RANDOM%
::
::

Here is the contents of the QSF.bat (Quickstream Fix) script:

@ECHO OFF
::
::
:: Quickstream Fix script to run MPG and/or TP recordings through VideoRedo's Quickstream Fix
:: (Last Edited 03-22-07)
::
::
IF %~x1==.tp SET EXT=/t4
IF %~x1==.mpg SET EXT=/t1
::
:: To keep MPG files from being passed through QSF, uncomment the line, below...
::IF %~x1==.mpg GOTO ENDQSF
:: To keep TP files from being passed through QSF, uncomment the line, below...
::IF %~x1==.tp GOTO ENDQSF
::
:CHECKFORQSFMPG1
IF EXIST "%~d1\*.MPG" GOTO WAITTOQSFMPG1
GOTO ENDCHECKFORQSFMPG1
:WAITTOQSFMPG1
ECHO Another Quickstream Fix job is running (MPG file exists in "%~d1\")
ECHO Waiting 60 seconds...
SLEEP 60
GOTO CHECKFORQSFMPG1
:ENDCHECKFORQSFMPG1
::
:CHECKFORQSFTP1
IF EXIST "%~d1\*.TP" GOTO WAITTOQSFTP1
GOTO ENDCHECKFORQSFTP1
:WAITTOQSFTP1
ECHO Another Quickstream Fix job is running (TP file exists in "%~d1\")
ECHO Waiting 60 seconds...
SLEEP 60
GOTO CHECKFORQSFTP1
:ENDCHECKFORQSFTP1
::
:CHECKFORCUT1
IF EXIST "%~d1\*.VPRJ" GOTO WAITTOCUT1
GOTO ENDCHECKFORCUT1
:WAITTOCUT1
ECHO Another cutting job is running (VPRJ file exists in "%~d1\")
ECHO Waiting 60 seconds...
SLEEP 60
GOTO CHECKFORCUT1
:ENDCHECKFORCUT1
::
ECHO .
ECHO Quickstream Fix will be run on "%~nx1" and output to "%~d1\%~nx1"
ECHO .
ATTRIB -R "%~pdnx1"
cscript //nologo "c:\program files\videoredoPlus\vp.vbs" "%~pdnx1" "%~d1\%~nx1" %EXT% /d
::
:TRYTOMOVE1
IF EXIST "%~pdnx1" ATTRIB -R "%~pdnx1"
MOVE /Y "%~d1\%~nx1" "%~pdnx1"
IF EXIST "%~d1\%~nx1" GOTO CANNOTMOVE1
GOTO MOVED1
:CANNOTMOVE1
ECHO Cannot overwrite "%~pdnx1", so waiting for 10 minutes to try again...
SLEEP 600
GOTO TRYTOMOVE1
:MOVED1
ECHO .
::
:ENDQSF
::
::

Here is the contents of the Comskip.bat script:

@ECHO OFF
::
::
:: Comskip script to commercial detection on Videos, using Comskip
:: (Last Edited 03-22-07)
::
::
SET COMSKIPFOLDER=M:\MEDIA\Files_Backup\Comskip
::
::
IF EXIST "%~pdn1*log*" DEL "%~pdn1*log*"
IF EXIST "%~pdn1*txt" DEL "%~pdn1*txt"
IF EXIST "%~pdn1*index.dat" DEL "%~pdn1*index.dat"
IF EXIST "%~pdnx1.chapters.xml" DEL "%~pdnx1.chapters.xml"
IF EXIST "%~pdn1.chp" DEL "%~pdn1.chp"
IF EXIST "%~pdn1.vprj" DEL "%~pdn1.vprj"
ECHO .
ECHO Running Commercial Detection, using Comskip
ECHO .
"%COMSKIPFOLDER%\comskip.exe" "%~pdnx1"
::
:: clean up log and txt files
IF EXIST "%~pdn1*log*" DEL "%~pdn1*log*"
IF EXIST "%~pdn1*txt" DEL "%~pdn1*txt"
IF EXIST "%~pdn1*index.dat" DEL "%~pdn1*index.dat"
::
::

There is one line in the Comskip.bat that will need to be changed; the 7th line that says “SET COMSKIPFOLDER=M:\MEDIA\Files_Backup\Comskip.” That drive and path should be changed to whatever your real path is to your Comskip folder that includes your comskip.exe and comskip.ini.

Here is the contents of the Cut.bat script:

@ECHO OFF
::
::
:: Cut commercials script using the VPRJ file from Comskip and running it through VideoRedo.
:: (Last Edited 03-22-07)
::
::
IF %~x1==.tp SET EXT=/t4
IF %~x1==.mpg SET EXT=/t1
::
IF EXIST "%~pdn1.VPRJ" GOTO CHECKMATCHING
GOTO ENDCUT
::
:CHECKMATCHING
COPY "%~pdn1.VPRJ" %~d1\
::
::
::
::
IF EXIST "%~d1\*Modern*Marvels*.VPRJ" GOTO CUTNOW
IF EXIST "%~d1\*MythBusters*.VPRJ" GOTO CUTNOW
IF EXIST "%~d1\*Monk*.VPRJ" GOTO CUTNOW
IF EXIST "%~d1\*How*It*Made*.VPRJ" GOTO CUTNOW
IF EXIST "%~d1\*Star*Trek*Voyager*.VPRJ" GOTO CUTNOW
IF EXIST "%~d1\*Weird*U*S*.VPRJ" GOTO CUTNOW
IF EXIST "%~d1\*Scrubs*.VPRJ" GOTO CUTNOW
IF EXIST "%~d1\*Psych-(*.VPRJ" GOTO CUTNOW
::
::
::
::
:: If no matching files are found, we delete the temp VPRJ file and exit
DEL "%~d1\%~n1.VPRJ"
GOTO ENDCUT
::
:CUTNOW
::
:CHECKFORQSFMPG2
IF EXIST "%~d1\*.MPG" GOTO WAITTOQSFMPG2
GOTO ENDCHECKFORQSFMPG2
:WAITTOQSFMPG2
ECHO Another Quickstream Fix job is running (MPG file exists in "%~d1\")
ECHO Waiting 60 seconds...
SLEEP 60
GOTO CUTNOW
:ENDCHECKFORQSFMPG2
::
:CHECKFORQSFTP2
IF EXIST "%~d1\*.TP" GOTO WAITTOQSFTP2
GOTO ENDCHECKFORQSFTP2
:WAITTOQSFTP2
ECHO Another Quickstream Fix job is running (TP file exists in "%~d1\")
ECHO Waiting 60 seconds...
SLEEP 60
GOTO CUTNOW
:ENDCHECKFORQSFTP2
::
:CHECKFORCUT2
DEL "%~d1\%~n1.VPRJ"
IF EXIST "%~d1\*.VPRJ" GOTO WAITTOCUT2
GOTO ENDCHECKFORCUT2
:WAITTOCUT2
ECHO Another cutting job is running (VPRJ file exists in "%~d1\")
ECHO Waiting 60 seconds...
SLEEP 60
GOTO CUTNOW
:ENDCHECKFORCUT2
::
COPY "%~pdn1.VPRJ" %~d1\
ECHO .
ECHO Cutting file, using VideoRedo
ECHO .
cscript //nologo "C:\Program Files\VideoReDoPlus\vp.vbs" "%~pdn1.VPRJ" "%~d1\%~nx1" %EXT% /d
ECHO .
SLEEP 1
DEL "%~d1\%~n1.VPRJ"
IF EXIST "%~pdn1.VPRJ" DEL /F "%~pdn1.VPRJ"
::
:TRYTOMOVE2
IF EXIST "%~pdnx1" ATTRIB -R "%~pdnx1"
MOVE /Y "%~d1\%~nx1" "%~pdnx1"
IF EXIST "%~d1\%~nx1" GOTO CANNOTMOVE2
GOTO MOVED2
:CANNOTMOVE2
ECHO Cannot overwrite "%~pdnx1", so waiting for 10 minutes to try again...
SLEEP 600
GOTO TRYTOMOVE2
:MOVED2
IF EXIST "%~pdnx1.chapters.xml" ATTRIB -R "%~pdnx1.chapters.xml"
IF EXIST "%~pdnx1.chapters.xml" DEL /F "%~pdnx1.chapters.xml"
ECHO .
::
:ENDCUT
IF EXIST "%~pdn1.chp" DEL /F "%~pdn1.chp"
::
::

In this script on lines 19-26, I have some name-matching rules set up. These are my rules that determine what shows get the commercials cut out. To delete a rule, just delete that whole line or comment it out with double colons ( :: ). To put in your own rule, copy and paste one of my lines and then enter your own name-matching information. Keep in mind that this is variable-matching, so you should not have spaces and you can substitute an asterisk as a wild card. All of them should end with .VPRJ because that is the file it is actually using for matching.

For all of these scripts to work, you should NOT have any MPG, TP, or VPRJ files on the root of your video drive. On my system, my video drive is drive V: with many folders. The QSF and CUT scripts actually output video to the root of the drive, as a temporary location. They also check the root of the drive, to see if MPG, TP, or VPRJ files already exist there. This is done as a crude method of determining whether another QSF or CUT job is already running. I try to limit the total number of concurrently-running tasks, to minimize overall system I/O. This isn't necessarily required, but I do a lot of recording and compressing, so my system has a lot of I/O, as it is.

All of these scripts can be run independently or combined into one large script. If you don't need to run DELAY or QSF or CUT, you can leave out whatever parts you don't need and the remaining scripts should work just as well.

There are a few recommended settings within VideoRedo. Go into Tools -> Options -> Stream Parameters and set Detect/Resync Missing Frames to "Resync- Insert Extra Video Frames". While here, make sure Double Video memory is checked and set Encoder Quality to High. On the Enhanced Plus page, check "Reduce CPU Priority While Saving". On the Transport Stream Output page, change Output Mux Rate to Manual and set Mbps to 30.000. I also usually change Abort After "5" Buffer Underflows, though I'm not entirely sure how necessary this is.

*EDIT* Edited Comskip.ini settings for new version (79_98) which is significantly faster than the old version and appears to be more accurate.

*EDIT 2* If you enjoy using Comskip, please consider sending the program author a donation. (Paypal erik@kaashoek.com) He deserves it!

*EDIT* Edited Comskip.ini settings for version 79_111.

*EDIT* Edited Comskip.ini settings for version 79_125.

*EDIT* Edited with fix from post 129 and for Comskip.ini settings for Comskip 79_129.

[NickHark]

Bravo!

[healy]

Amazing amount of info.
I have been trying to figure out how to set up comskip with BTV & VRD. This goes a long way to achieving that goal. (Now I just need to understand what I need to do. )

I have a reluctance to trust a software to actually cut commercials accurately. Can your script be modified to merely mark for later manual editing?

Thanks for all your work and especially with sharing it with the rest of us.

--Gene

[cmcquistion]

Originally Posted by healy

Amazing amount of info.
I have been trying to figure out how to set up comskip with BTV & VRD. This goes a long way to achieving that goal. (Now I just need to understand what I need to do. )

I have a reluctance to trust a software to actually cut commercials accurately. Can your script be modified to merely mark for later manual editing?

Thanks for all your work and especially with sharing it with the rest of us.

--Gene

All you have to do is copy and paste my examples into one long file (using notepad), then save that file as a batch file (batch.bat, for example) and set it up.

The Cut.bat section of my script has variables you can fill in with particular show names that you want to cut. If your show isn't listed, there, then it isn't cut. You can comment out the examples I have, in there, with double colons, and those will be ignored. If there is no name matching, in that section, then your video does NOT get cut. It only gets the previous steps applied to it (fixing video and running Comskip.)

In my Comskip.ini, one of the options that I have turned on is to output a VideoRedo file (in addition to the BTV smartchapter file). This VPRJ file is associated with the original video. You can use this file to cut the video, manually, if you desire. You just double-click the file, tell Windows what program to open it with (VideoRedo), and when VideoRedo opens that VPRJ file, it opens the associated video file and overlays where the commercial markings are. At that point, you just click File Save As and save the video file. While it is saving, VideoRedo will cut out those regions, highlighted in red. This is how you can manually cut the files.

My script example, above, at the end of the Cut.bat section has a line that reads:

"IF EXIST "%~pdn1.VPRJ" DEL /F "%~pdn1.VPRJ""

What this does is delete the VPRJ file, at the end of the all these processes, so it isn't just left out there. If you want to leave that file, so you can use it for manual cutting, just delete this line or comment it out with double colons. That will leave the VPRJ file in place. The only problem with this is that Beyond TV will not delete these files, after you've watched and deleted a show, so they will start to pile up. Personally, I use Dirmon2 to watch my video folders and clean up these leftover files, when no MPG or TP file exists, any more. In my case, I output CHP files (ZoomPlayer chapter files) also, so Dirmon2 cleans those up, also.

I'll post a screenshot, below, of my Dirmon2 Global Settings screen that has all the information you need to set up a "cleanup job" using Dirmon2.

By the way, if you don't necessarily want to use these scripts "live" by setting up a CustomPostProcessing step and everything, you can test these out by launching them from the command line (the manual way) or just drag and drop a video file from your BTV folder onto the icon for the batch file (batch.bat, comskip.bat, etc) or the icon for the shortcut to the batch file. This will launch a command prompt and run the batch file on that video file.

[Jakesty]

cmcquistion: What a great write-up! I hope I have the time to implement your solution. This info prompted me to give you an idea for improvement I think you'd like to consider.

The command below will pull a line out of a text file that you could use to store those shows you want recorded. That way the BAT file isn't being edited and you could just store this on the desktop or where ever.

For /F "tokens=*" %%i in (c:\batch\Cut-shows.txt) Do c:\batch\BAT-RUN.bat %%i
(all on one line of course)
It would have to be modified at the "DO" statement to perform based on your criteria/shows to cut.
Then when BAT-RUN.bat operates in the file you use %1 to reference the %%i name that it pulled from the file. In your case you might just use %%i and call a subroutine in the bat file.
I have used this to process a command against a list in that file in which I had servers. Does this give you any help? Just a thought and I wanted to contribute.
Jake

[bits]

cmcquistion: Nice work, thank you!

What is the reason for doing the QSF? I thought Videoredo does that during the cutting.

Also, is there a way to specify which PID VRD will QSF? In my neck of the woods the ts show I want is often not the first PID.

Thanks again.

[ZetaVu]

I still would like BTV to give us the ability to edit commercial cuts (smartchapter or whatever) on the fly.

Picture this, post processing automatically marks what it thinks are commercial breaks. As I watch the file (with its attached xml commercial file) if I skip to a commercial point and it is wrong, I can click a button on my remote and get the option to add or delete a start/stop point. I currently do this with Videoredo before saving a new file sans commercials, basically I open the smartchapter file as a project, then skip to commercial points to make sure they are accurate, replacing them if needed. It would be great to be able to do this in BTV, and then send a process to videoredo or even to showsqueeze for archival. Specifically keeping the metadata etc. Then I could take the commercial free mpeg to make a DVD or save the commercial free DIVX.

[cmcquistion]

Originally Posted by Jakesty

cmcquistion: What a great write-up! I hope I have the time to implement your solution. This info prompted me to give you an idea for improvement I think you'd like to consider.

The command below will pull a line out of a text file that you could use to store those shows you want recorded. That way the BAT file isn't being edited and you could just store this on the desktop or where ever.

For /F "tokens=*" %%i in (c:\batch\Cut-shows.txt) Do c:\batch\BAT-RUN.bat %%i
(all on one line of course)
It would have to be modified at the "DO" statement to perform based on your criteria/shows to cut.
Then when BAT-RUN.bat operates in the file you use %1 to reference the %%i name that it pulled from the file. In your case you might just use %%i and call a subroutine in the bat file.
I have used this to process a command against a list in that file in which I had servers. Does this give you any help? Just a thought and I wanted to contribute.
Jake

Thanks for the idea. I'll have to give this some time to wrap my brain around this and test it.

Originally Posted by bits

cmcquistion: Nice work, thank you!

What is the reason for doing the QSF? I thought Videoredo does that during the cutting.

Also, is there a way to specify which PID VRD will QSF? In my neck of the woods the ts show I want is often not the first PID.

Thanks again.

I do the QSF to fix up the video stream, whether it is a TP file of MPG. The TP files generally need it the most. I do the MPG files, because I can. I noticed that commercial generation ~can be~ more accurate on a file that has been cleaned up, first. This depends on how messed up the original file is, of course.

VideoRedo does do QSF during cutting, but I don't cut all shows. I only cut a handful of shows, regularly. Going back to my previous point, I want to have a good, smooth video to get the best possible commercial detection and markings, especially if I'm going to use that information to cut the file.

Unfortunately, there isn't a way to specify what PID VRD will use (at this time). In my particular location, it picked the correct PID for every channel (OTA HD) except for an NPT subchannel (not the main NPT). The only way to know what PID the VideoRedo will pick is to try it out. There is some logic built into VideoRedo for picking the correct channel, but I don't know exactly how it works. I'm now using a HDHomeRun, in addition to my OTA HD. The HDHomeRun only records the appropriate PID. It does not record the entire channel, like OTA HD recording, so with the HDHR, it is never a problem because only one program exists in the stream.

[bits]

@cmcquistion:

Looks like it is time to pickup the HDHomerun and build a new PC. My current system could handle your suggested setup but the CPU would be pegged an awful lot of the time. I am planning on building either a Core 2 EX6800 Conroe or a Core 2 Quad Q6600 Kentsfield around September of this year. Either should be able to handle the multiple cpu intensive tasks.

One potential issue to solve is how to get the finished file to the correct drive and folder. I have video scattered across (5) drives and 10-15 folders. But this is a minor issue that can be handled manually until when ever!

Thanks again for the hard work!!

[cmcquistion]

Originally Posted by bits

One potential issue to solve is how to get the finished file to the correct drive and folder. I have video scattered across (5) drives and 10-15 folders. But this is a minor issue that can be handled manually until when ever!

The way my script works, currently, is that it puts the final file (whether it is cut or not) back in the original folder that it came from. I do this so automatic Showsqueeze jobs can run, properly, and I can implement show number limits. The script could be easily modified to output the files to another location, though, like a different drive or network attached storage. By doing this, you lose the ability to have automatic showsqueeze, and implement show number limits (like recording a maximum of 5 episodes of a certain show.). Of course, some people won't be bothered by these limitations, so it may not be a big deal, for them.

Regarding your current hardware, you should be able to run this on your current system, just fine, provided you have 1 GB of memory. My last system had an Athlon 3800+ (dual core) and 1 GB of RAM. My current system has a Core 2 Duo E4300 (the cheapest Core 2 Duo) and 2 GB of RAM, though I very rarely go over 1 GB of RAM usage and usually this only happens if a program has crashed. I also use this as my main system, so I always have multiple other applications running and hogging up memory (like Firefox).

[acegolfer]

CMC,

Thanks.

I'm recording 2 OTA HD every night and would like to automate the following process.

After recording in TP,
1. comskip.exe,
2. cut commercial,
3. encode in Xvid (using quality = 50%)

I can follow your batch files for steps 1 and 2. What do I need to do for Xvid encoding? I found you participated in autoxvid thread. Have you automated this process?

[cmcquistion]

Originally Posted by acegolfer

CMC,

Thanks.

I'm recording 2 OTA HD every night and would like to automate the following process.

After recording in TP,
1. comskip.exe,
2. cut commercial,
3. encode in Xvid (using quality = 50%)

I can follow your batch files for steps 1 and 2. What do I need to do for Xvid encoding? I found you participated in autoxvid thread. Have you automated this process?

Yeah, I've been playing with AutoXvid to do the encoding part, after my scripts are finished. The combination is pretty straightforward. Just insert another step in the CustomPostProcessing.xml file to launch AutoXvid.exe after my script (or scripts, if you break them apart.)

Here is an example"

<CustomTasks>
<Task>
<GUID>531E0175-87F4-4d6f-B464-E603BC641DAE</GUID>
<Priority>Deferred</Priority>
<PropertyCollection>
<Property>
<Name>Command</Name>
<Value>M:\MEDIA\Files_Backup\Comskip\batch.bat</Value>
</Property>
</PropertyCollection>
</Task>
<Task>
<GUID>531E0175-87F4-4d6f-B464-E603BC641DAE</GUID>
<Priority>Immediate</Priority>
<PropertyCollection>
<Property>
<Name>Command</Name>
<Value>C:\AutoXvid\AutoXvid.exe</Value>
</Property>
</PropertyCollection>
</Task>
</CustomTasks>

Here is a slighly modified version of my AutoXvid.ini file. You should be able to use this (just modify your video folders and such) to get what you're asking for, from AutoXvid:

[AutoXVID]
BTVRecordFolders= V:\VIDEO\
BTVRecurseFolders= V:\VIDEO\
OutPutSize= Quality 50%
RoundDuration=1
MediaTypes= mpg, tp
CutCommercials=
CutProgram= None
HCCutProgram= None
MediaTypes= mpg, tp
CutCommercials=
TV= NTSC
ComSkipClip= No
;SeriesCutExclude=
;SeriesEncodeExclude=
;ChannelCutExclude=
;ChannelEncodeExclude=
;NoEncodeAfterCut= No
Priority= Normal
MakeSubDir= No
;SubDirFormat= Genre\Title
Logging= Yes
Quiet= Yes
Voice=0
OutPutFolder=
AutoUpdate= No
ShowAutoGK= Yes
Encode= No

[Episode]
EpisodeData= Yes

[SubTitles]
SubTitlesDesired= No
BurnSubTitles= No
CCPath= .subtitles

[ScheduledRun]
AutoLookAhead=
AGKExecutionHours= Immediate
DeleteSource= No
ShutDown= No
WaitForSmartChapters= No

[cmcquistion]

I thought I would mention that I just looked at my Showsqueeze logs and for a one-hour MPG show, it only took 18 minutes to run Quickstream fix, then Comskip, then cut out the commercials. This is on my Intel Core 2 Duo E4300, which is overclocked to 3 GHz (@ 1.425 Vcore). None of these programs (VideoRedo or Comskip) are multi-threaded, so they only use one CPU Core, effectively. I can run two jobs at once and see only a very small performance penalty.

TP files, of course, take longer than MPG files to process because of their higher bitrate and resolution. If I had to guess, I would say that they typically take 2-3 times as long as MPG files.

VideoRedo is faster than Snapstream's Streamsnip, for trimming Transport Streams (and VideoRedo's Quickstream fix does more actions and does them better.) VideoRedo runs very quickly on MPG files, typically taking less than 2 minutes for a one-hour MPG show. Comskip, likewise, is significantly faster than Snapstream's Smartchapter generation, while doing a more accurate job and offering the abillity to tweak it's settings.

[GeorgeH]

Not to offend ---

VideoReDo is not a very good program to use for correcting "errors." As evidence ...

I had been having problems ShowSqueezing some MPG files (analog recordings from VCR tapes using BeyondTV). One particular problem was that ShowSqueeze reduced 3hr programs to 1 to 1-1/2 hr. Clearly the problem must be "errors" in the digital file.

I tried VideoReDo (Quickstream Fix) on the MPG files and the "fixed" MPG files came out to be about the same length 1 to 1-1/2 hrs. "Proof" that the files had errors.

Not so fast. I had several 3 hr MPG files that when ShowSqueeze came out 3 hr (WMP format). Ran those MPG files through VideoReDo (Quickstream Fix). Came out 1 to 1-1/2 hr long. Clearly VideoReDo does not work out of the box.

While I am sure that with enough time one can make VideoReDo work on some files, VideoReDo is not the solution claimed.

The amazing part of this is ALL the original MPG files play perfectly with a number of different software packages - including BeyondTV and VideoReDo.

---

I did contact the VideoReDo people with a great offer if they could make their product work but ...

[ZetaVu]

I have to disagree here. I consider Videoredo the last resort in fixing a file before I give up. It will play back most corrupted formats, make them playable and correct audio sync issues. I took a 3+ hour tp of the NFC playoffs that picked up enough dropped frames to be corrupt in btv, ran a quickstream fix and poof, 3hr+ of playable HD video. I have never had a problem created by videoredo, and most problems it has fixed. Any problem it hasn't been able to fix nothing else has been able.

Now, I still use other software when building mpg files, but video redo and joiner are the best and quickest for cutting out or joining together.