Suggested XML Format

[kingargyle]

Well, while I'm waiting from some Videos to compress, I decided to take a look at the userprofile.xml that was created when I did a custom video recording. One thing I have noticed about the way .NET creates XML files is that unless the programmer really knows something about XML, it generates some pretty ugly looking XML. With all the Property and Name/Value tags in there, it really doesn't describe what it is the file represents.

An example is the following hacked up extract from userprofile.xml:
<SSPropsCollection><PropertyCollection>
<Property><Name>TwoPass</Name><Value>FALSE</Value></Property>
<Property><Name>Type</Name><Value>QUALITY_TYPE_STREAMING</Value></Property>
<Property><Name>VBREnabled</Name><Value>FALSE</Value></Property>
<Property><Name>VideoBitrate</Name><Value>1300000</Value></Property>
<Property><Name>VideoFourCC</Name><Value>827739479</Value></Property>
<Property><Name>VideoFPKS</Name><Value>30000</Value></Property>
</PropertyCollection></SSPropsCollection>

Not very descriptive unless you know what you are looking for, and pretty much not human readable, which is one of XMLs big benefits.

Now compare it to something like the following:

<SnapStream>
<CustomVideoSettings name="Test Custom1" guid="5b3d7cf2-816b-4e39-ac6e-15e29386af3c">
	 <AudioBitrate value="96040"/>
	 <AudioChannels value="2" />
	 <AudioFormatTag value="353" />
	 <AudioHz value="44100"/>
	 <EnableResize value="FALSE"/>
	 <EncodingWindow value="5000"/>
	 <MaxBufferWindow value="2000"/>
	 <Modified value="TRUE"/>
	 <PeakBitrate value="3000000"/>
	 <TwoPass value="FALSE"/>
	 <Type>QUALITY_TYPE_STREAMING</Type>
	 <VBREnabled value="FALSE"/>
	 <VideoBitrate value="1300000"/>
	 <VideoFourCC value="827739479">Windows Media V7</VideoFourCC>
	 <VideoFPKS value="30000"/>
	 <VideoDimensions height="480" width="640"/>
	 <VideoInputEqualsOutput value="TRUE<"/>
	 <VideoKeyframeInterval value="4"/>
	 <VideoQualityScale value="0"/>
	 <VideoTransformDeinterlace value="TRUE"/>
	 <VideoTransformInverseTelecine value="FALSE"/>
</CustomVideoSettings>
<CustomVideoSettings name="Test Custom2" guid="5b3d7cf2-816b-4e39-ac6e-15e29386af3c">
	 <AudioBitrate value="96040"/>
	 <AudioChannels value="2" />
	 <AudioFormatTag value="353" />
	 <AudioHz value="44100"/>
	 <EnableResize value="FALSE"/>
	 <EncodingWindow value="5000"/>
	 <MaxBufferWindow value="2000"/>
	 <Modified value="TRUE"/>
	 <PeakBitrate value="3000000"/>
	 <TwoPass value="FALSE"/>
	 <Type>QUALITY_TYPE_STREAMING</Type>
	 <VBREnabled value="FALSE"/>
	 <VideoBitrate value="1300000"/>
	 <VideoFourCC value="827739479">Windows Media V7</VideoFourCC>
	 <VideoFPKS value="30000"/>
	 <VideoDimensions height="480" width="640"/>
	 <VideoInputEqualsOutput value="TRUE<"/>
	 <VideoKeyframeInterval value="4"/>
	 <VideoQualityScale value="0"/>
	 <VideoTransformDeinterlace value="TRUE"/>
	 <VideoTransformInverseTelecine value="FALSE"/>
</CustomVideoSettings>
<SnapStream>

Personally I think this is much easier to understand than the prior example. You know right off the bat that this file is going to be dealing with CustomVideoSettings, and that it has two of them. The attributes specify the name and guid for each of these (yes the guid is the same, this is a mock up file after all). Underneath each of these CustomVideoSettings are the Elements thatmake up those settings. Much easier to read and interpret from a human stand point, but also easy for a machine to use as well.

It also allows for easier manipulation via XML based languages like XPATH and XSLT. It actually describes what makes up a CustomVideoSetting instead of just using generic Property/Name/Value tags. Plus if you want to get to a particular element quickly, you don't have to loop through the collection, but can do a quick Xpath to get the item you want (i.e.) If you were using XSLT and wanted to get all nodes that had TwoPass with a Value of FALSE, you would write something like:
/SnapStream/CustomVideoSettings[TwoPass = "FALSE"]

That would just return you a set of the nodes that matched that criteria, instead of having to search through the entire collection yourself. BTV uses the Property/Name/Value xml format all over the place, but unfortunately that approach is great if you are a machine, but not so nice if you are a human trying to figure out what you are working with.

Anyway, just some suggestions for the BTV developers to help us in the Dev Community. Thoughts and comments are always welcomed.

[flavius]

Typically those comments ought to be in the code generating the XML, cause that's what developers mess with. You need to understand the code, not the XML.

If you want to bypass the api, that's a whole different story, but not in terms what the XML is supposed to look like. -

[btvbuddy]

As a corporate developer with 14 years experience with clients like Reuters Data Group and IBM (to name a few) you have put the biggest grin on my face (see my avatar)

What were talking about is Data Mapping and Data Standards, of course these need to be version controlled.

Now, you have put together a data mapping between the SnapStream XML and our open Standard XML for Encoding Profiles. You have done an excellent start, congrats.

My application is focused on Standard XML data schema for

Content
ContentCategories
VideoRecording
VideoLibrary

I will go ahead and post my Standard Schemas later today, of course feedback is welcome.

I am hoping that together we can develop a set of cooperating integrated utilities and applications built around our Standards, no one is a island to themselves.

Great start, more on this later.

[kingargyle]

BTVBuddy, I also have about 12 years worth of experience doing Data Integration. Started off on the Mainframe with various EDI standards, and have worked my way into Object Oriented technologies using XML and XML Schemas (not used enough in my opinion). I'll be glad to toss what knowledge I have into your schemas.

My problem with the XML that the BTV API produces is that it isn't descriptive or meaningful, and for the aspect of having to know the code, yes you have to know the code, but the data that is coming in should be mapped in a way that is also human readable. There are many ways to manipulate the data, and an API is just one of them.

It's the old EDI problem surfacing again. If I have a DTE (date) segment in an EDI file, what does that DTE segment represent? Does it represent a Purchase Order Date, an Invoice date, etc. Where as you can become more specific with XML. <PurchaseOrderDate>2004-04-18</PurchaseOrderDate> is much more meaningful.

The same goes for the XML file I produced, by quickly looking at it, you know what you are dealing with. The BTV API could have easily been modified to produce the above with a very simple XSLT before it sent the output out to the client, and just as easily could have it mapped back to the internal representation it needs when receiving information. The whole purpose of a WebService is to make it easier to integrate with other systems, part of that is coming up with a good data exchange format that is easy for any language to implement. You also want to make groupings make sense. Generic use of Property/Name/Value do not necessarily represent what a particular item is, unless you dig down to the actual values within that tag. The example I provided, gives the same ability as the multiple Property/Name/Value elements, but in less space, and at a finer granularity. Just by glancing at the file (as a programmer) I have an easier time understanding what I'm looking at.

There are all sorts of opinions on this, but I feel if programmers are going to use XML for data representation, then they really need to use XML to it's fullest. Why use extra memory and resources, if you can stream line and give a clearer representation by tweaking just a bit?

Microsoft has made it easier for Webservices to be created, but I feel at the cost of truely understanding the underlying technologies they are using. Just because it produces XML doesn't mean that it is GOOD XML.

[flavius]

I have been in the business for 15 years now, this coming November. And only recently took a very close look at some of the X12N formats, and wrote a converter for the 837's which took almost four days. Now there is an XML representation that's friendly on your eyes, and 12 times the size of the 837. For some odd reason people like looking at that stuff. I don't get it. It's not persistent. It shouldn't be.

I still have to be convinced that any of the problems, the new web admin is showing, has anything to do with the XML, or the fact that's not too friendly on your eyes I'd recommend to focus on the problems.

Btw: Have you guys ever thought about who is going to own the IP to whatever you do? What if they pick a BSD-like license?

[kingargyle]

I agree, that XML is very bloated when compared to flat files or even EDI itself. It is why I don't believe it should be used in a Batch environment, it was never ment for that type of application. But for realtime Webservices transactions that are fairly small, then it is a good choice. I had this same debate several years ago with the Electronic Commerce line of business I was with (all die hard EDI developers).

The web admin itself isn't where I'm coming from, I'm looking at data exchange between applications. Whether that be from the Web Service, or just parsing the XML files directly. I don't think changing the layout will fix ANY of the Web Admin problems as those are more design and useability UI issues (besides the fact that 3/4 of the time the server just craps out on the pages themselves with .NET execution errors).

When you take a look at it from an interoperability/data exchange standpoint, you have SAGE TV doing one thing, Meedio TV doing something completely different, Beyond Media doing something else, GotTV doing something else. Everyone of these products uses their own file formats, and if you wanted to import or export information from one system to the other, you have to write custom coding for each one. BTVBuddy's idea of a standard markup for Video Content exchange would help to make one standard set of API's or Web Services that each of these platforms could conform to. Thus you could in theory, write a Web Admin for BTV or SageTV without having to change your own code, as they would use a standard API to talk with each other, because they are all following a standard file specification. I know this is a dream, but hey, we are allowed to dream.

I agree, that SnapStream has much more pressing bugs/design flaws to solve. However, standardization would help ease the pain of integration and importing data from extrenal programs, plus give the user an easy way to move from one system to another if they wanted to switch software, or used in another application. We are all experienced programmers, so we are always going to have different opinions.

[btvbuddy]

Excellent flow, with regards to license, never sign anything without an attorney present and a case of Jack Daniels! Next, on the issue of Standards - this is a lot to swallow and we need a specialist on board to handle this (we may not have the right people on board and of course that will slow things down a bit) What we could consider is ISO HTPC Standards, but hey I am not going to bark up that tree right now (WAY too busy)

http://isotc.iso.org/livelink/liveli...22608&vernum=0

Food for thought.

Just a reminder we should agree on how things are going to be release in the interim to an official group decision - myself I may just say my Standards are released via either Freeware or GNU Lesser License (not really sure)

If SnapStream want to kill the chance of having anyone onboard with their SDK all they do to do is setup a EULA (the current BeyondTV SDK is an example) that basicaaly says we work for free and SnapStream owns all IP.

Oh well. that's my .02c