markreidvfx / pyaaf2

Read and write Advanced Authoring Format (AAF) files

pyaaf2

A python module for reading and writing Advanced Authoring Format (AAF) files. pyaaf2 is a rewrite of pyaaf1 in pure python.

Features

• Read/Write AAF files
• Modifying existing AAF files inplace
• Embedding DNxHD/DNxHR/WAV media
• Copying objects between files
• Low level read/write Compound File Binary access
• Lazy file reading
• Zero dependencies, does not use AAF SDK

Requirements

• Python >= 2.7

Installation

You can install pyaaf2 via:

pip install pyaaf2

or if you want to use the latest development git master:

git clone https://github.com/markreidvfx/pyaaf2
cd pyaaf2
python setup.py install

Documentation

Documentation is available on Read the Docs.

TODO

• More docs
• More tests
• More helper classes
• Port more pyaaf1 examples
• MXF linking improvements
• AMA linking improvements
• XML support

pyaaf2's Issues

component_at_time is missing

Hi there Mark.

Our implementation of walk() calls a function called component_at_time. It'd be great to have this functionality back. See the commented lines here...

Do you think we can get this implemented? Thank you!

def _walk_item(thing):
    slot = thing.slot
    if not slot:
        return
    segment = slot.segment
    if isinstance(segment, aaf2.components.SourceClip):
        yield segment
        for item in _walk_item(segment):
            yield item
    # elif isinstance(segment, aaf2.components.Sequence):
    #     clip = segment.component_at_time(thing.start_time)
    #     if isinstance(clip, SourceClip):
    #         yield clip
    #         for item in clip._walk_item():
    #             yield item
    #     else:
    #         raise NotImplementedError(
    #            "Sequence returned {} not implemented".format(type(segment))
    elif isinstance(segment, aaf2.components.EssenceGroup):
        yield segment
    elif isinstance(segment, aaf2.components.Filler):
        yield segment
    else:
        raise NotImplementedError(
            "walking {} not implemented".format(type(segment))
)

Add metadata to custom columns and leave tape name blank?

Hi Mark!

I have two related questions regarding metadata.

First question is regarding the ability to write metadata to custom fields for the Avid. Is there an ability to write metadata to a custom column that can be chosen via the bin menu inside the Avid? If so, would you be able to point me in the right direction to implement that?

Second, is it possible to leave the tape name field empty when creating an embedded sequence aaf? If I leave it null it defaults to the sequence_name.#.

Thanks!

parse out mxf files

hi mark!!

i can't find my last comment i made last year about this topic, sorry! is there any way for me to parse out .mxf files with this python version of pyaaf? or output some kind of human readable version of the aaf file that i can parse myself?

thank you!

Example request

Successfully ran the example in the docs, and I'm wondering if you could show an example of placing multiple audio clips at various TC addresses in a single track. Trying to grasp the right syntax relative to the AAF object spec...Thanks

link_external_mxf doesn't work for audio-only mxf files

I'm trying to link mxf wrapped PCM audio streams. I've tried using aaf2's built-in method link_external_mxf and also pyaaf's example link_external_mxf.py, but it only seems to work for video files. How would I link audio mxf files similar to video mxf files?

Implement property and object name aliases

Some extension properties have been renamed throughout the lifetime of AAF. Allow properties the have multiple names for compatibility.

MediaStreamPluginGUID - > MediaContainerGUID

This one was just seems name wrong in MC files

CommentMarkerUSer -> CommentMarkerUser

Also some objects have spaces in the names??

"Avid MC Mob Reference"

Information on external mxf linking and metadata

Hi

First of all, thanks for this library. For the project I'm working on the server is not able to use the AMT SDK (OS type) so having a pure python version is incredibly useful.

My use case is that on a server I'm going to build the files that need delivering to a workstation (aaf and associated external dnxhd mxf files). I've done some preliminary tests with linking external mxfs and that seemed to work. However, where we build the aaf file, and where the files get consumed are two different locations so that paths to the mxf files will be very different for the user vs what is on the server. My understanding is that the aaf file will contain to the path to the mxf when it was created so this aaf file won't be able to work with MC. Is this correct?

Admittedly my understanding of the AAF format is very limited so my understanding maybe completely off.

What I'd like to do is create an aaf file that contains references to mxf files as they appear to the end user.

Also, I'd like to include locator information in the aaf file. Is this possible? Are there any examples?

Handling Unicode

Hi Mark,

Hit some more unicode snags from our AAF adapter:

advanced_authoring_format.py", line 56, in _walk_item
    slot = thing.slot
  File "/home/steinbach/sw/src/pyaaf2/aaf2/components.py", line 145, in slot
    slot_id = self.slot_id
  File "/home/steinbach/sw/src/pyaaf2/aaf2/components.py", line 126, in slot_id
    return self['SourceMobSlotID'].value
  File "/home/steinbach/sw/src/pyaaf2/aaf2/properties.py", line 145, in value
    return self.typedef.decode(d)
  File "/home/steinbach/sw/src/pyaaf2/aaf2/types.py", line 121, in decode
    assert len(data) == self.size
  File "/home/steinbach/sw/src/pyaaf2/aaf2/types.py", line 95, in size
    return unpack('B', data)[0]

And:

advanced_authoring_format.py", line 220, in _transcribe
    metadata["Length"] = item.length    
  File "/home/steinbach/sw/src/pyaaf2/aaf2/components.py", line 23, in length                                                                           
    return self['Length'].value                                     
  File "/home/steinbach/sw/src/pyaaf2/aaf2/properties.py", line 145, in value                                                           
    return self.typedef.decode(d)                                     
  File "/home/steinbach/sw/src/pyaaf2/aaf2/types.py", line 704, in decode                                                               
    return self.renamed_typedef.decode(data)
  File "/home/steinbach/sw/src/pyaaf2/aaf2/types.py", line 121, in decode                                                                               
    assert len(data) == self.size                                   
  File "/home/steinbach/sw/src/pyaaf2/aaf2/types.py", line 95, in size                                                                  
    return unpack('B', data)[0]         
TypeError: Struct() argument 1 must be string, not unicode ```

Any suggestions?

Create AAF with references to mxf op atom to use with avid interplay

Hi,

I use pyaaf2 to create an aaf file which links to OP Atom MXF files. I use this AAF file to checkin new media files to Avid Interplay. The code looks like this:

from pathlib import Path
import aaf2

data_folder = Path('//example/workspace/Avid MediaFiles/MXF/folder/')

first_mxf = data_folder / '1315LY_v1.mxf'
second_mxf = data_folder / '1315LY_a1.mxf'
third_mxf = data_folder / '1315LY_a2.mxf'
forth_mxf = data_folder / '1315LY_a3.mxf'
fifth_mxf = data_folder / '1315LY_a4.mxf'

out_file = "test2.aaf"
with aaf2.open(out_file, 'w') as f:
    f.content.link_external_mxf(first_mxf)
    f.content.link_external_mxf(second_mxf)
    f.content.link_external_mxf(third_mxf)
    f.content.link_external_mxf(forth_mxf)
    f.content.link_external_mxf(fifth_mxf)

This is just sample code for simple testing.
The AAF is generating all fine.

I use the Interplay Web Services API from Avid to check this AAF file into interplay. This is working without errors. I am also able to get this asset from Interplay Access into Avid Media Composer. Video and Audio is online.

There is only 1 problem:
Interplay Access shows some metadata like "media size" and "file locations". The "file locations" show all referenced media files (so all OP Atom files).

The checked-in AAF from pyaaf generates a masterclip which is shown in Interplay Access. But masterclip has media size 0 and no file locations. The problem now is, that I can't delete the referenced media files within Interplay Access, because it does not "see" the referenced MXF files.

All Masterclips which are generated by other programs (commercial transcoders, Avid MC,..) have those properties set..

Does anybode know why this happens or has some ideas about this?

I know this is a very specific question but maybe I am doing something wrong with linking the mxf's using pyaaf2?

Thanks for any hints!

Question - AAF version

Can you please tell me what AAF format version does this library comply with?
Also, would it generate Legacy objects similarly to pyaaf1?

Thanks and keep up the awesome work!!

Seperate Avid Extensions from base AAF 1.1 data model

Currently the base data model has everything thing in it

Set StartTime of filler, or sequence?

Hi! Thanks for an amazing module.

The answer to this question might be obvious, and I hope it is ok to ask questions here, but I can't find a way to set my sequence start time, or the start time of a filler for that matter. I thought that the EventSlotOrigin would be able to set the start time, but I cant get it to work in Pro Tools. If I open the AAF in Media Composer and export it straight away it ends up on 01:00:00:00 as I want it to, but when comparing all props between the two aafs I can't find the difference.

Very much appreciate any help.
Rather simplified my code looks like this:

import aaf2

with aaf2.open("newaaf.aaf", 'w') as f:
    ems = f.create.EventMobSlot()
    ems['EditRate'].value = 25
    ems['EventSlotOrigin'].value = 9000 #as a test value, does not work
    ems['SlotID'].value = 1000
    ems['PhysicalTrackNumber'].value = 1

    sequence = f.create.Sequence("DescriptiveMetadata")
    marker = f.create.DescriptiveMarker()
    marker['DescribedSlots'].value = set([1])
    marker['Position'].value = 100
    marker['CommentMarkerUser'].value = "A new marker that ends up in PT Comment-field"
    sequence.components.append(marker)

    ems.segment = sequence
    mob = f.create.CompositionMob()
    p_slot = mob.create_sound_slot()
    filler  = f.create.Filler("sound", 0)
    p_slot.segment.components.append(filler)
    p_slot['PhysicalTrackNumber'].value = 1
    mob.slots.append(ems)
    f.content.mobs.append(mob)

Best regards
Martin Sandström

AAF files with embedding footage not working on Avid Media Composer 6.5.4 nor 8.5.3

Hi i'm just starting playing with this and everything seems pretty exiting!
Thanks for this effort!

Sadly my Avid is complaining with the resulted aaf files, even with the example from the docs:
http://pyaaf.readthedocs.io/en/latest/#embedding-footage-example

Those are my first steps so its most surely that i'm missing some technical details either in the transcoding or the structure of the aaf.

Following the example exactly produce an aaf file that loads the clips in the avid, but not the media, logging this error in the console:

Clip (internal): Demo2.PHYS
    Minor Change: Error with media reference (clip will be offline).

Error importing media for 'Demo2'. (FileMob 1 of 2, VIDEO) Error: 'DIDReader::XraiseUnexpectedStreamStatus async file system error.'
Error importing media for 'Demo2'. Error: 'CM_INCOMPAT_TRACK_TYPES' 

I'm attaching the aaf file.
example2.zip

Create Offline Clip to relink in Avid

Hi Mark! Thanks so much for your help so far. One more big thing I'd like to accomplish is to create clips in a sequence aaf that imports as offline clips in the Avid. The idea is that I can mix and match clips on a sequence that have embedded media, as well as some that are offline clips, so that I can relink these offline clips to already imported media in the Avid, i.e. new clips vs old.

I have a feeling that basically it's the same process as creating embedded media clips but instead of importing dnxhd essence into the sourcemob, I need to create CDCI descriptors for the sourcemob. I've been able to create a sequence with offline clips but it imports into the avid with an error (CM_LABEL_NOT_UNIQUE) and the offline clips don't have tape names or timecodes so I can't relink them to the previously imported clips.

Would you be able to point me to an example of how I can specify a clip to have all the attributes of a dnxhd 36 master clip but just not have any actual media embedded? Thanks!

WAVEDescriptor implementation/PR

Hi there, I've been playing around with AAFs for Pro Tools like @will-riley in #2 , and I have working code for creating useable MOBs for linking to WAV files through WAVDescriptors. I had the thought of submitting this as a PR, would you like that and how exactly would you like the interface to look?

Extracting sound from SoundMasterTrack

I have the option Add Audio mixdown set in Avid, so that I get a "master" audio track on export. I need help extracting the audio from this. Having done a dump of all mobs in f.content.mobs, I have this:

   <aaf2.mobslots.TimelineMobSlot 8 at 0x7f642a92e368>
     EditRate <Rational TypeDefRecord> 24
     Origin <aafPositionType TypeDefRename> 0
     Segment <kAAFTypeID_SegmentStrongReference TypeDefStrongRef>
       Components <kAAFTypeID_ComponentStrongReferenceVector TypeDefVarArray>
         <aaf2.components.Filler Filler at 0x7f642a798418>
           DataDefinition <DataDefinitionWeakReference TypeDefWeakRef> <aaf2.dictionary.DataDef SoundMasterTrack 0e040101-0101-0000-060e-2b3404010101 at 0x7f642a942520>
           Length <aafLengthType TypeDefRename> 235
       Length <aafLengthType TypeDefRename> 235
       DataDefinition <DataDefinitionWeakReference TypeDefWeakRef> <aaf2.dictionary.DataDef SoundMasterTrack 0e040101-0101-0000-060e-2b3404010101 at 0x7f642a942520>
     Segment <kAAFTypeID_SegmentStrongReference TypeDefStrongRef> <aaf2.components.Sequence Sequence at 0x7f642a942418>
     PhysicalTrackNumber <aafUInt32 TypeDefInt> 1
     SlotID <aafUInt32 TypeDefInt> 8
     SlotName <aafString TypeDefString> 

I don't know how to go from that to opening the essence, can you assist please

PyAAF2 performance seems to be about 4x slower than PyAAF1

I've been looking at the performance of PyAAF2 vs PyAAF1, and it unfortunately looks like PyAAF2 is about 4x slower. For example, this file:
https://github.com/PixarAnimationStudios/OpenTimelineIO/blob/master/opentimelineio_contrib/adapters/tests/sample_data/linear_speed_effects.aaf

Takes about 4x as long to read with PyAAF2. Profiling suggests a couple things might be going on but a clear winner doesn't obviously jump out to me...

Here is the script I was using to profile (via the OTIO adapter):

#!/usr/bin/env python
import cProfile
import StringIO
import pstats
import opentimelineio as otio

pr = cProfile.Profile()
pr.enable()
tl = otio.adapters.read_from_file(
    "opentimelineio_contrib/adapters/tests/sample_data/linear_speed_effects.aaf"
)
pr.disable()
s = StringIO.StringIO()
sortby = 'cumulative'
# sortby = 'tottime'
ps = pstats.Stats(pr, stream=s).sort_stats(sortby)
ps.print_stats()
print s.getvalue()

Any suggestions or thoughts on what might be done to improve performance in PyAAF2?

Thanks!

Writing corrupt AAF files

On MacOS 10.12.6.

I am having difficulty creating AAF files that can be imported into Media Composer or Premiere. I have tried even the simplest implementation:

with aaf2.open("test.aaf", 'w') as f:
    f.save()

I have also tried this unit test without luck.

I can read the AAF data with pyaaf, but Media Composer won't open it and Premiere crashes.

Ama linking on Media Composer won't work unless you close and reopen bin

Not sure if this is a issue with Media Composer or pyaaf2

Exporting session to Pro Tools

I'm trying to generate a Pro Tools wav only session without success.

Here is my code:

import subprocess
import json
import aaf2

def probe(path, show_packets=False):

    cmd = ['ffprobe', '-of','json','-show_format','-show_streams', path]

    if show_packets:
        cmd.extend(['-show_packets',])

    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

    stdout,stderr = p.communicate()

    if p.returncode != 0:
        raise subprocess.CalledProcessError(p.returncode, subprocess.list2cmdline(cmd), stderr)

    return json.loads(stdout.decode('utf8'))

with aaf2.open("test.aaf", "w") as f:
    path = "/path/to/file.wav"
    meta = probe(path)
    f.content.link_external_wav(meta)

Unfortunately Pro Tools display an error message when trying to open it:

Could not complete your request because an unexpected error happened while reading the file's main composition sequences. 

I'm trying to investigate it but do you know if there is something wrong with what I do?

I tried check what's wrong by generating a very simple session:

protools.zip

Unfortunately when trying to open it with examples/qt_aafmodel.py it is empty and I have the following log:

Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 136, in rowCount
    return parentItem.childCount()
  File "examples/qt_aafmodel.py", line 28, in childCount
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 136, in rowCount
    return parentItem.childCount()
  File "examples/qt_aafmodel.py", line 28, in childCount
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'
Traceback (most recent call last):
  File "examples/qt_aafmodel.py", line 158, in parent
    parentItem = childItem.parent()
  File "examples/qt_aafmodel.py", line 55, in parent
    self.setup()
  File "examples/qt_aafmodel.py", line 104, in setup
    self.references.sort()
TypeError: '<' not supported between instances of 'MobID' and 'MobID'

Multitrack Video AAF in Resolve

First of all, thank you Mark for this amazing resource.

I wrote a simple script with pyaaf2 that creates an embedded sequence AAF with multiple video tracks that imports into the Avid as expected, and am trying to import that same AAF in Resolve.

When I try to import the pyaaf2 created AAF into Resolve, Resolve says that the AAF doesn't have a primary timecode track. I was able to get past that by creating an empty sequence slot and setting the start timecode to 86400 (1hr at 24fps, although it sets the seq duration to 1036800 in the Avid).

The bigger issue is that it then asks me to choose a Source Timeline and gives me a dropdown option to choose one of the tracks I've created when I select on, Resolve only shows me that single track on the timeline.

Can you give me some guidance on how I can get Resolve to import all video tracks I've created into a single timeline? I'm thinking that I need to contain the tracks in a NestedScope but I'm clueless as to how I can get that to work.

I'm attaching my janky script. I realized that Resolve doesn't support embedded AAF's so I'll have to link to the files instead of embedding but that's another issue for another day.

Thanks!
aaf_avid.txt

The audio file in AAF does't showed in Protools

Firstly, Thank you for your resource.

I am trying to learn how to put audio file by code. I follow the Quickstart "Embedding Footage" part to embed a wav file into the aaf. But when I import that aaf into Protools, the audio file doesn't show up. It just become a block on the track. But the size of the aaf seems contain the wav.

here is the screenshot.

`
import aaf2

with aaf2.open("example2.aaf", 'w') as f:

# objects are create with a factory
# on the AAFFile Object
mob = f.create.MasterMob("Demo2")

# add the mob to the file
f.content.mobs.append(mob)

edit_rate = 25

# lets also create a tape so we can add timecode (optional)
tape_mob = f.create.SourceMob()
f.content.mobs.append(tape_mob)

timecode_rate = 25
start_time = timecode_rate * 60 * 60 # 1 hour
tape_name = "Demo Tape"

# add tape slots to tape mob
tape_mob.create_tape_slots(tape_name, edit_rate,
                           timecode_rate, media_kind='picture')

# create sourceclip that references timecode
tape_clip = tape_mob.create_source_clip(1, start_time)

# now finally import the generated media
#mob.import_dnxhd_essence("sample.dnxhd", edit_rate, tape_clip)
mob.import_audio_essence("/Users/yedianyang/Desktop/AD_001.wav", edit_rate)

`

Documentation on how to access the timeline

I'm trying to iterate over the timeline and get all the start and end timecodes of every clip in the sequence. Is this achievable with this package and if so, how would that be accomplished? Thank you!

unpack_from doesn't work on bytearray in python <= 2.7.3

Release 1.0.0 works, but not current master branch.
ta

Traceback (most recent call last):
  File "./test_pyaaf2.py", line 8, in <module>
    with aaf2.open("./b0000.aaf", "r") as f:
  File "/tools/SITE/python/aaf2/file.py", line 188, in __init__
    self.cfb = CompoundFileBinary(self.f, self.mode, sector_size=sector_size)
  File "/tools/SITE/python/aaf2/cfb.py", line 750, in __init__
    self.mini_stream_chain = self.get_fat_chain(self.root.sector_id)
  File "/tools/SITE/python/aaf2/cfb.py", line 434, in sector_id
    sid = struct.unpack_from(str('<I'), self.data, 116)[0]
TypeError: unpack_from() argument 1 must be string or read-only buffer, not bytearray

Cannot import_rawvideo_essence with mp4. Pixel layout not encoding

I am trying to the the SourceMob method import_rawvideo_essence. After some debugging I think the issue may be due to pixel_layout parameter data type being incorrect. I've tried many different types and cannot get it to work.

Assuming the 'sample.mp4` is in the same directory, the following code can reproduce the issue.

import aaf2

with aaf2.open("example2.aaf", 'w') as f:

    # objects are create with a factory
    # on the AAFFile Object
    mob = f.create.MasterMob("Demo2")

    # add the mob to the file
    f.content.mobs.append(mob)

    edit_rate = 25

    tape_mob = f.create.SourceMob()
    tape_mob.import_rawvideo_essence(
        path="sample.mp4",
        edit_rate=edit_rate,
        width=480,
        height=360,
        pixel_layout=(4, 3),
    )

From my debugging attempts, the error is coming from aaf2/types.py:682 when attempting to encode the pixel_layout argument here:

value = typedef.encode(data[key])

From the above snippet, the data object is an integer of value 4, therefore the error shows the following:

File "/usr/local/lib/python3.7/site-packages/aaf2/mobs.py", line 247, in import_rawvideo_essence
    descriptor['PixelLayout'].value = pixel_layout
  File "/usr/local/lib/python3.7/site-packages/aaf2/properties.py", line 55, in func_wrapper
    result = func(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/aaf2/properties.py", line 162, in value
    self.data = self.typedef.encode(value)
  File "/usr/local/lib/python3.7/site-packages/aaf2/types.py", line 354, in encode
    result += element_typedef.encode(item)
  File "/usr/local/lib/python3.7/site-packages/aaf2/types.py", line 680, in encode
    value = typedef.encode(data[key])
TypeError: 'int' object is not subscriptable

What do I need to provide for pixel_layout to import the mp4 as an essence?

MarkIn-MarkOut

I'm trying to set MarkIn and MarkOut points on imported dnxhd essence.

Here's what is not working:

master_mob = f.create.MasterMob()
slot = master_mob.import_dnxhd_essence("sample.dnxhd", 24)
slot['MarkIn'].value = 1
f.content.mobs.append(master_mob)

How should I be doing this?

Mix and match embedded and already imported media in AAF

Hi Mark!

I was hoping you might be able to point me in the right direction. I'm having success embedding media into aaf's to load into the avid. This works great if all the clips in the sequence are new versions that the avid hasn't imported before. What I'm hoping to do is mix and match new embedded media as well as clips that link to already imported clips in the avid without having to import that media again.

If I wanted to specify that a clip is already in the avid and have it link to that media in the timeline, how should I go about doing that in pyaaf2?

Thanks again for all your help and an amazing resource!

Can't read AAF

The read AAF example doesn't seem to work for me, with no discernible errors.

import aaf2

with aaf2.open("my_aaf.aaf", "r") as f:

    # get the main composition
    main_compostion = next(f.content.toplevel())

    # print the name of the composition
    print(main_compostion.name)

    # AAFObjects have properties that can be
    # accessed just like a dictionary
    print(main_compostion['CreationTime'].value)

    # video, audio and other track types are
    # stored in slots on a mob object.
    for slot in main_compostion.slots:
        segment = slot.segment
        print(segment)

Traceback (most recent call last):
  File "read.py", line 8, in <module>
    main_compostion = next(f.content.toplevel())
StopIteration

Saving as xml

In the original pyaaf, there was a saving option to go to xml (as shown in the example below from (markreidvfx/pyaaf#6). Is this no longer available in this version?

aafFile = aaf.open(filename, 'r')
aafFile.save('/tmp/del.xml')
aafFile.close()

Large filesize

I'm creating an AAF with a single embedded instance, which is a dnxhd file containing a single frame. The dnxhd file is 184kb, created with the following command:
ffmpeg -f lavfi -i testsrc=size=1920x1080 -v error -y -nostdin -an -c:v dnxhd -pix_fmt yuv422p -b:v 36M -framerate 24 -frames:v 1 sample.dnxhd

I've then used a trimmed down version of the example "Writing an AAF" code:

import aaf2

if __name__ == '__main__':
	with aaf2.open('out.aaf', 'w+') as f:
		master_mob = f.create.MasterMob()
		master_mob.import_dnxhd_essence("sample.dnxhd", 24)
		f.content.mobs.append(master_mob)

The resultant AAF is 12Mb. Is there anything I can do to reduce this size? Please note that I require to embed the media and cannot keep it external.

Using pyaaf2 from a C++ project

I develop in C++/Qt and I was wondering if you had an idea how to use your library from my C++ project?

pip install pyaaf2?

It would be awesome if you could pip install pyaaf2 from https://pypi.org/

Should NestedComponent have a slots property?

Hi Mark,

Thanks for this great bit of work.

Should NestedScope component have a Slots property? Hardly a showstopper since nestedScope['Slots'] works fine.

I've been converting the qt_timeline code.

Thank you,
RF

Error link_external_mxf when using opatom video

Hi,

i am a little unexperienced otherwise i would have loved to open a PR.
Got this error message when using ama_link or link_external_mxf.
I believe line 721 in mxf.py should look like this:

FALSE: n['slot_id'].value = self.data['URLString']
CORRECT: n['URLString'].value = self.data['URLString']

My code:

with aaf2.open(sys.argv[1]+".aaf", 'w') as f:
    f.content.link_external_mxf("C:\\dev\\python\\opatom\\GUMO_1906V01C478F96E.mxf")

Message:
Traceback (most recent call last):
File "C:\dev\python\create_linked.py", line 22, in
f.content.link_external_mxf("C:\dev\python\opatom\GUMO_1906V01C478F96E.mxf")
File "C:\Python27\lib\site-packages\aaf2\content.py", line 75, in link_external_mxf
return m.link(self.root)
File "C:\Python27\lib\site-packages\aaf2\mxf.py", line 831, in link
mobs.append(package.link())
File "C:\Python27\lib\site-packages\aaf2\mxf.py", line 262, in link
mob.descriptor = d.link()
File "C:\Python27\lib\site-packages\aaf2\mxf.py", line 587, in link
d['Locator'].append(item.link())
File "C:\Python27\lib\site-packages\aaf2\mxf.py", line 721, in link
n['slot_id'].value = self.data['URLString']
File "C:\Python27\lib\site-packages\aaf2\core.py", line 330, in getitem
raise KeyError(key)
KeyError: u'slot_id'

"pct_parser" not implemented in PyAAF2

I am attempting to re-implement the example from PyAAF to "dump avid titles" and could not locate the equivalent function 'pct_parser' in the new module. Is there a reason this has not been added?

Copying the code from PyAAF appears to work as intended (in my specific example). Just wanted to check if there was a known limitation or reason as to why it wasn't ported yet considering it is pure python in the original version

Error when walking AAF: Walking <class 'aaf2.components.OperationGroup'> not implemented

NotImplementedError on aaf2.components.OperationGroup on an AAF exported from Avid MediaComposer 2018.11

Call is technically coming from OpenTimelineIO, however it appears that the error is being generated by the AAF2 module.

Traceback (most recent call last): File "~/read_aaf.py", line 18, in <module> main(aaf) File "~/read_aaf.py", line 11, in main timeline = otio.adapters.read_from_file(aaf_file) File "~\lib\site-packages\opentimelineio\adapters\__init__.py", line 144, in read_from_file **adapter_argument_map File "~\lib\site-packages\opentimelineio\adapters\adapter.py", line 136, in read_from_file **adapter_argument_map File "~\lib\site-packages\opentimelineio\plugins\python_plugin.py", line 128, in _execute_function return (getattr(self.module(), func_name)(**kwargs)) File "~\lib\site-packages\opentimelineio_contrib\adapters\advanced_authoring_format.py", line 919, in read_from_file result = _transcribe(storage, parent=None, editRate=None, masterMobs=masterMobs) File "~\lib\site-packages\opentimelineio_contrib\adapters\advanced_authoring_format.py", line 229, in _transcribe child = _transcribe(mob, item, editRate, masterMobs) File "~\lib\site-packages\opentimelineio_contrib\adapters\advanced_authoring_format.py", line 236, in _transcribe track = _transcribe(slot, item, editRate, masterMobs) File "~\lib\site-packages\opentimelineio_contrib\adapters\advanced_authoring_format.py", line 339, in _transcribe child = _transcribe(item.segment, item, editRate, masterMobs) File "~\lib\site-packages\opentimelineio_contrib\adapters\advanced_authoring_format.py", line 328, in _transcribe child = _transcribe(component, item, editRate, masterMobs) File "~\lib\site-packages\opentimelineio_contrib\adapters\advanced_authoring_format.py", line 333, in _transcribe item, metadata, editRate, masterMobs File "~\lib\site-packages\opentimelineio_contrib\adapters\advanced_authoring_format.py", line 676, in _transcribe_operation_group child = _transcribe(segment, item, editRate, masterMobs) File "~\lib\site-packages\opentimelineio_contrib\adapters\advanced_authoring_format.py", line 328, in _transcribe child = _transcribe(component, item, editRate, masterMobs) File "~\venv\lib\site-packages\opentimelineio_contrib\adapters\advanced_authoring_format.py", line 249, in _transcribe mobs = _find_timecode_mobs(item) File "~\lib\site-packages\opentimelineio_contrib\adapters\advanced_authoring_format.py", line 112, in _find_timecode_mobs for c in item.walk(): File "~\lib\site-packages\aaf2\components.py", line 207, in walk type(segment))) NotImplementedError: Walking <class 'aaf2.components.OperationGroup'> not implemented

How to read Start time and length from the nested source clip on a audio track

from source_clip.data.media_kind I get to know its a Sound, and I now want to get the StartTime
so I do source_clip["StartTime"] it gives me 0 but if I open the same in otioview I can see the start time in frames format .Any help will be greatly appreciated.

Null or invalid characters in some string values

We've seen a few instances of corrupt-looking strings in some string values.

In the example AAF file linked on this OTIO issue: AcademySoftwareFoundation/OpenTimelineIO#474 we get this value for AvidASPIPlugInName:

Time Compression Expansion\u0000\u0000\uf37e\u1029

but aafDump shows:

Time Compression Expansion

Performance issue linking external wavs

I'm using pyaaf2 to create AAF files that can contain as many as about 5k-10k short clips, using aaf.content.link_external_wav(metadata). This seems to be causing pyaaf2 some trouble. It can take as much as 30 minutes for my computer to link them all. I ran a performance test using gprof2dot which gives a visualization of where the processing time is being spent.

It seems that linking 5k wav files is leading to 15k calls to aaf2.properties.extend (for each of the 3 mobs), and I think that creating a tree this large is causing a problem.

Do you have any suggestions for how this could be optimized?

reference in requirements.txt

Hi
When adding pyaaf2 to requirements.txt, it works using:
git+http://github.com/markreidvfx/pyaaf2.git

Is there a way to specify version / releases in any way when adding it?
Thanks!

Embedding media

I'm trying to create an AAF with embedded media and a sequence.

The code in this test is what I'd like to do, but with embedded media, like in this example.

Here is what I have so far, which is not throwing errors, is creating a sequence, but has 'media offline':

import aaf2

outputfile = "test.aaf"
edit_rate = 24
clip_length = 20

inputs = [
    (24.0,  "test1", "test.dnxhd"),
    (24.0, "test2", "test2.dnxhd")
]

if __name__ == "__main__":
    with aaf2.open(outputfile, "w+") as f:
        comp_mob = f.create.CompositionMob()
        sequence = f.create.Sequence(media_kind="picture")

        timeline_slot = comp_mob.create_timeline_slot(edit_rate)
        timeline_slot.segment = sequence

        f.content.mobs.append(comp_mob)

        for timecode_rate, clip_name, video_file in inputs:
            tape_mob = f.create.SourceMob()
            tape_slot, tape_timecode_slot = tape_mob.create_tape_slots(clip_name, edit_rate, float(timecode_rate))
            tape_slot.segment.length = clip_length

            f.content.mobs.append(tape_mob)

            file_mob = f.create.SourceMob()

            file_description = f.create.CDCIDescriptor()

            file_description['ComponentWidth'].value = 8
            file_description['HorizontalSubsampling'].value = 4
            file_description['ImageAspectRatio'].value = '16/9'
            file_description['StoredWidth'].value = 1920
            file_description['StoredHeight'].value = 1080
            file_description['FrameLayout'].value = 'FullFrame'
            file_description['VideoLineMap'].value = [42, 0]
            file_description['SampleRate'].value = edit_rate
            file_description['Length'].value = clip_length

            file_mob.descriptor = file_description

            clip = tape_mob.create_source_clip(slot_id=1, length=clip_length)
            slot = file_mob.create_picture_slot(edit_rate)
            slot.segment.components.append(clip)

            f.content.mobs.append(file_mob)

            master_mob = f.create.MasterMob()

            clip = file_mob.create_source_clip(slot_id=1, length=clip_length)
            slot = master_mob.create_picture_slot(edit_rate)
            slot.segment.components.append(clip)

            # ???
            tape = tape_mob.create_source_clip(1, length=clip_length)
            master_mob.import_dnxhd_essence(video_file, edit_rate, tape=tape)
            # ???

            f.content.mobs.append(master_mob)

            clip = master_mob.create_source_clip(slot_id=1, length=clip_length)
            sequence.components.append(clip)

This is pretty much all just copied from example code; the two lines surrounded by question marks are my poor guessing attempt at importing the media.

Many thanks

Cannot parse mxf style aaf

Hi, thanks for this great library!
To be honest i don't know why there exist 2 totally different aaf file styles, one MS Compound file type and the other more like MXF style. However, the AAF C++ API seems to support both. Should pyaaf2 also support both? ..example file attached.

aaf_mxf_style.zip

Compatibility for python < 2.7.7 ?

Hi

During my tests with embedded media i found an issue with audio.py when running under a python version lower that 2.7.7:

  File "exportForEditorial.py", line 309, in aaf_import_media_source
    asource_mob = master_mob.import_audio_essence(stream['path'], sample_rate)
  File "/soft/python/packages/aaf2/mobs.py", line 173, in import_audio_essence
    source_slot = source_mob.import_audio_essence(path, edit_rate, tape)
  File "/soft/python/packages/aaf2/mobs.py", line 281, in import_audio_essence
    a = audio.WaveReader(path)
  File "/soft/python/packages/aaf2/audio.py", line 42, in __init__
    wave.Wave_read.__init__(self, f)
  File "/soft/python/2.7/lib/python2.7/wave.py", line 163, in __init__
    self.initfp(f)
  File "/soft/python/2.7/lib/python2.7/wave.py", line 143, in initfp
    self._read_fmt_chunk(chunk)
  File "/soft/python/packages/aaf2/audio.py", line 48, in _read_fmt_chunk
    wFormatTag, self._nchannels, self._framerate, dwAvgBytesPerSec, wBlockAlign = struct.unpack_from('<HHLLH', chunk.read(14))
TypeError: Struct() argument 1 must be string, not unicode

I found that this is because a change in the pack/unpack modules from 2.7.6 to 2.7.7 and also because the use of unicode_literals from future:
http://python-future.org/stdlib_incompatibilities.html#struct-pack

I really need to support this since we have several environments here, so I just did a patch with python-future (future.utils.bytes_to_native_str) as described here: http://python-future.org/stdlib_incompatibilities.html#array-array

Of course this require to validate the python version to define the proper value on each one.
Apparently is the only place, after that patch everything was working fine also with video but I certainly didn't run the tests, maybe tomorrow.

I can maintain the patch on my own, but if there is a chance to have that supported in the main repo will be awesome!

Modifying "Tape" metadata

Hello,
Could you please provide an example on how to to modify the "Tape" (or "Source File") metadata of assets within a pre-existing AAF?

Thank you very much for this impressive tool!

Extracting Image Data

Im trying to get access to embedded images inside an AAF. Is it possible to read the 'Picture' data on a SourceClip?

Here is my mess for some guidance...

import aaf2

with aaf2.open("my.aaf", "r") as f:
    for mob in f.content.mobs:
        print(mob.name, mob.usage, mob.comments)
        for slot in mob.slots:
            print(slot)
            print(slot.edit_rate)
            print(slot.origin)
            segment = slot.segment
            print(segment)
            print(segment.start)
            print(segment.media_kind)

➜  pythontest python read.py
(None, None, <aaf2.misc.TaggedValueHelper object at 0x105f3e7d0>)
<aaf2.mobslots.TimelineMobSlot 1 "Test" at 0x105f39520>
24
0
<aaf2.components.SourceClip SourceClip at 0x105a06d60>
0
Picture
(u'??????', None, <aaf2.misc.TaggedValueHelper object at 0x105e64d10>)
<aaf2.mobslots.TimelineMobSlot 1 "A Slot" at 0x105f39af8>
24
0
<aaf2.components.SourceClip SourceClip at 0x105f39b50>
0
Picture

trying to migrate from pyaaf -questions

I'm trying to migrate from pyaaf but since you are trying to do thing a bit different with this version I am not sure how to do a few things. The first one is trying to create a custom OpDef. Here is how I'm doing it in pyaaf. I can't seem to figure out how to do it in this version.

    # CREATE PARAMETER AVIDEFFECTID DEFINITION
    typedef = f.dictionary.lookup_typedef("Rational")
    param = aaf.define.ParameterDef(f, parmID, "AvidEffectID", "avid effect id", typedef)
    f.dictionary.register_def(param)

    # CREATE MATTE KEY DEFINITION
    op_def = aaf.define.OperationDef(f, effectID, "MatteKey_2", "matte key def")
    op_def.media_kind = "picture"
    op_def['IsTimeWarp'].value = "False"
    op_def['Bypass'].value = 2
    op_def['NumberInputs'].value = 3
    op_def['OperationCategory'].value = "OperationCategory_Effect"

    f.dictionary.register_def(op_def)

    # Added Parameter Definition to Operation Definition
    op_def.add_parameterdef(param)

    # CREATE OPERATION GROUP
    opgroup = aaf.component.OperationGroup(f, "picture", length, op_def)

Second thing is how do you add comments. Here is the code from pyaaf.

 if attributes:
        for attribute, content in attributes.items():
            mastermob.append_comment(attribute, content)

Thanks

About AMA linking

Hi

I'm struggling a little with the ama linking and AVID, specially because I'm dealing with an old version (6.5.4)

I'm trying to replicate the structure that AVID used when it export its own AAF files, where if the media was linked as "ama" an import of that file properly relink the media again. But all my attempts end with offline media on Avid.

I indeed see some differences between the file from avid and the one that I'm getting from pyaaf2:

• In pyaaf2 descriptors use a "MediaContainerGUID" while onthe Avid file I see a "MediaStreamPluginGUID", for this I found that you was previously using that:
  0d526d1#diff-2f38143c704c632c6111a64d8c2a6b82
  So not sure which one is the correct for my case, for now I modified pyaaf2 to use the the MediaStreamPluginGUID with the UUID from the AVID file.

• There are a lot of TaggedValues in the Avid file, some of them pointing to AMA values, but there is one that I'm not sure how to replicate, the "_IMPORTSETTING" which have a value "__AttributeList" and a bunch of other TaggedValues linked to it.

• There is also the differences in attributes for the CDCI and PCM Descriptors, but I'm hoping that those not necessary affect the "ama" linking behavior.

Right now I'm using the create_ama_link method to create the whole thing, but Im thinking that I will need to do it manually...

aaf_file = aaf2.open(output_aaf, "w")
source_path = "s01-ep02.scd020_030_editorial_v001.mov"

meta = probe(source_path)
# Note that I modified create_ama_link to use the MediaStreamPluginGUID 
master_mob, src_mob, tape_mob = aaf_file.content.create_ama_link(source_path, meta)

master_mob.comments.append(aaf_file.create.TaggedValue("_IMPORTED_AMA", 1))
master_mob.comments.append(aaf_file.create.TaggedValue("_AMA_METADATA_COUNT", 0))
master_mob.comments.append(aaf_file.create.TaggedValue("Video File Format", "MOV"))
master_mob.comments.append(aaf_file.create.TaggedValue("Video", "Photo - JPEG"))
master_mob.comments.append(aaf_file.create.TaggedValue("Audio Format", "LPCM"))
master_mob.comments.append(aaf_file.create.TaggedValue("Field Ordering", "Default"))

src_mob.comments.append(aaf_file.create.TaggedValue("_VERSION", 2))
src_mob.comments.append(aaf_file.create.TaggedValue("_AMA_FILE_DATE_TIME", 1483575123))

tape_mob.comments.append(aaf_file.create.TaggedValue("_VERSION", 2))

aaf_file.save()
aaf_file.close()

Here you can find some diagrams:

From PyAAF2

From Avid

And both AAF files attached as zip
aafs.zip

Hopefully you have a better idea of how to aproach this. Thansk in advaned!

Easiest way to add audio to sequence level AAF's?

Hi Mark!

What would be the easiest way to include an audio track that is attached to each video clip to a sequence level AAF? At the moment, I am importing the audio essence to a mastermob in addition to importing the dnxhd essence. This gives me the sourceclip with a video and audio track in the avid bin but after adding them to a sequence on the CompositionMob slot, the resulting sequence AAF only has a video track and no audio. I'm assuming that I need to create an audio sequence in the CompositionMob as well but I'm unsure of when to do that and how to ensure that the video and audio are linked in the sequence in the Avid. Do I need to add the mastermob sourceclip to both a video and audio sequence, in a sense iterating through the shot and filler list twice? If I do this, I can create a sequence AAF with a video track and an audio track but the video and audio for a source comes in as 2 separate pieces of media and move in the timeline independently.

bottom line is that I would like to place clips with both video and audio into a sequence level AAF with filler. And I would like these to be from a single source clip so that they move on the timeline together. If I create a single shot AAF and not a sequence AAF, the clip in the bin has both video and audio but not if I create a sequence AAF.

Thanks!

Can I reference an external media file?

Hi! First of all, thank you for a wonderful library!

Does pyaaf2 support referencing external media files? That is, referencing a file that would be alongside the AAF? Right now I am importing the essence, but this isn't working out for us performance-wise.

Thanks for your time and hard work on this!

TimeCode creation

Hi

I'm migrating from pyaaf1

on pyaaf1, specifying length and position was in the constructor:
timecode = dictionary.create.Timecode(length, tc_position, fps)
on pyaaf2, i am able to only specify fps
timecode = f.create.Timecode(fps)

My questions is - how can i set the length and position for a newly created Timecode?

Thanks!