Media SDK Webinar 2014

Media SDK 1.1
Mikhail Mironov
AMD Media SDK Software Architect
Oct 22, 2014

MEDIA SDK USE CASES
Video Editing/Transcode Cloud Gaming Wireless Display
Video Conferencing Remote Desktop Video Playback
MEDIA SDK 1.1 AMF 2 WEBINAR | OCTOBER 22, 2014

MEDIA SDK – VALUE TO DEVELOPERS
Use AMD’s differentiated multimedia IP
• SW IP : GPU optimized video pre/post processing
• HW IP : VCE video encoder
• HW IP: UVD – video decoder
• HW IP : AMF-DEM library in latency sensitive applications like
wireless display and remote desktop
Optimal use of Heterogeneous Compute in Multimedia Apps
• DX to OpenCL inter-op
• SDK samples show developers how to build optimal video
processing pipelines
Developer Friendly
• Support for industry standard MFT APIs
• AMF - OS and framework agnostic C++ APIs
• Next – VAAPI support for Linux media developers

MEDIA SDK SUMMARY
 MFTs for leveraging Media HW accelerator blocks for video
encoding/decoding
 Video Quality MFT for video pre/post processing
 AMF: C++ API for media processing
 Support for Windows 7/8.1

MEDIA FOUNDATION COMPONENTS (MFTS)
 Access Video Compression Engine (VCE) for video
encoding
 Access Universal Video Decoder (UVD) for video
decoding
 Use GPU compute shaders for video processing
 Released with Media SDK 1.0

AMD MEDIA FRAMEWORK (AMF)
 C++ API and framework for media processing on AMD platforms
 OS agnostic : Initial support on Windows (Desktop and Windows Store) but
can be easily ported to other OSes
 Can be easily adapted to multimedia framework of choice – MFT, Gstreamer,
OpenMAX …
 Accept input from and provide inter-op with DirectX, OpenCL and OpenGL
 Replacement for OpenVideo
 Targeted for Media SDKv1.1

MEDIA SDK SOFTWARE STACK
AMD
Frameworks
Applications
OMX MFT
AMD Media Framework (AMF) (C++)
Drivers
AMD HW (UVD, VCE, GPU)
MFT
DirectX
OpenCL OpenGL
ISV
ISV
Native C/C++
ISV
Decoder Converter Processor Encoder
DEM Encoder
AMD
AMD
AMD
AMD AMD AMD
AMD

A10-7XXX
A10-Micro 6XXX
AMD Radeon™
R9 290
MEDIA SDK ROADMAP
Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2
2014
Media SDKv1.0:
- Video Quality MFT
for video processing
- AMF-DEM library
for low latency apps
- Windows 8/8.1
Store Support
2015
Media
SDK v1.0
GA
Media SDKv1.1:
- AMF for video
- encoding
- decoding
- Cloud Gaming
/VDI support
Planned
In
Planning
AMD Radeon™
R9 285
Media
SDKv2.0
Media
SDK v1.1
Beta
Media
SDK v1.1
GA
Media SDKv2.0:
- Video Processor
- Linux support
- VAAPI & AMF
- Open & closed
source Linux driver
Now
2016
Aug Dec
Media
SDKv3.0
Media SDKv3.0:
- Android support
- Audio support
- Camera support
- File containers (mux,
demux)
 

AMF COMPONENTS
 Full Decoder:
‒ UVD - H264 AVC, H264 VC1, WMV9, MPEG2, MPEG4 P2, H264 MVC, MJPEG, H265 HEVC
‒ Shader based MJPEG for legacy hardware
 Encoder: VCE: H264 AVC, SVC
 Video Converter (Shader based)
‒ Color Space Conversion, Scale, Composition
 Video Processor (Shader based)
‒ Steady Video, Color Space Conversion, Scaling, Brightness, Contrast, Saturation, Tint,
Vibrancy, Skin Tone Correction, Gamma, Brighter Whites, De-interlacing, Super
Resolution, Dynamic Contrast, Dynamic Range, Edge Enhancement, De-noise, Mosquito
Noise Reduction, False Contour Reduction, De-blocking, Demo Mode, FRC
 AMF DEM Screen capture encoder (VCE)
‒ Desktop only
 Capability Manager

AMF ARCHITECTURE - BASICS
 Elementary data types
All elementary data types are defined to make code portable to other OSs. Examples:
typedef __int64 amf_int64;
typedef __int32 amf_int32;
typedef size_t amf_size;
typedef void* amf_handle;
 C++ interfaces
Interfaces are implemented in the form of abstract (pure) C++ classes. Each interface is
derived from the AMFInterface basic interface. It exposes methods for reference
counting and querying for other interfaces :
class AMFInterface
{
public:
virtual amf_long AMF_STD_CALL Acquire()=0;
virtual amf_long AMF_STD_CALL Release()=0;
virtual AMF_RESULT AMF_STD_CALL QueryInterface(const AMFGuid&
interfaceID, void** ppInterface) = 0;
};

AMF ARCHITECTURE – INTERFACE DIAGRAM
AMFInterface
AMFPropertyStorage
AMFData AMFProperySorageEx
AMFSurface AMFBuffer AMFComponent
AMFContext
AMFPlane
1
1..*
Component
Surface

AMF ARCHITECTURE – PROPERTY STORAGE INTERFACE
 AMFPropertyStorage implements a property map between a string ID and an AMFVariantStruct
structure. There is a set of helper classes and template functions to safely manipulate
AMFVariantStruct in C++.
class AMFPropertyStorage : virtual public AMFInterface
{
public:
virtual AMF_RESULT SetProperty(const wchar_t *pName, AMFVariantStruct
value)=0;
virtual AMF_RESULT GetProperty(const wchar_t *pName, AMFVariantStruct*
pValue)=0;
virtual bool HasProperty(const wchar_t *pName)=0;
virtual amf_int32 GetPropertyCount() const = 0;
virtual AMF_RESULT GetPropertyAt(amf_int32 index, wchar_t* name, amf_size
nameSize, AMFVariantStruct* pValue) const = 0;
virtual AMF_RESULT Clear()=0;
virtual AMF_RESULT AddTo(AMFPropertyStorage *dest, bool bOverwrite, bool
bDeep)=0;
virtual AMF_RESULT CopyTo(AMFPropertyStorage *dest, bool bDeep)=0;
template<typename _T>
AMF_RESULT SetProperty(const wchar_t *pName, const _T& value);
template<typename _T>
AMF_RESULT GetProperty(const wchar_t *pName, _T* pValue);
};

AMF ARCHITECTURE – PROPERTY STORAGE EX INTERFACE
AMFPropertyStorageEx adds property description and validation features to
AMFPropertyStorage.
class AMFPropertyStorageEx : virtual public AMFPropertyStorage
{
public:
virtual amf_size GetPropertiesInfoCount() = 0;
virtual AMF_RESULT GetPropertyInfo(amf_size szInd,const AMFPropertyInfo**
pInfo) = 0;
virtual AMF_RESULT GetPropertyInfo(const wchar_t *pName,const
AMFPropertyInfo** pInfo)=0;
virtual AMF_RESULT ValidateProperty(const wchar_t *pName, AMFVariantStruct
val, AMFVariantStruct* outValidated) = 0;
};

AMF ARCHITECTURE - MEMORY
 Memory
AMF provides two types of memory objects: AMFBuffer and AMFSurface. Each object
implements the AMFData common base interface. Each object is allocated via factory
methods in the AMFContext interface. In addition, the AMFContext interface provides
methods to create AMFBuffer or AMFSurface wrapping existing memory objects from
various APIs:
• DX9 surfaces (Win Desktop only)
• DX11 Textures (Win Store and Win Desktop)
• OpenCL 2D images and buffers (Win Desktop only)
• OpenGL textures (Win Desktop only)
• Host memory
 GetNative()
AMFPlane and AMFBuffer interfaces provide the GetNative() method. It allows to
access the underlying memory object allocated by one of the technologies listed
above. This method always return pointer to void. This call does not change the
reference count of reference-counted objects (derived from the IUnknown interface or
OCL’s memobj)
 Thread safety
All memory objects in AMF are not thread-safe (except AMF reference counting). It is
assumed that any memory object is used by only one component at a time.

AMF ARCHITECTURE – DATA INTERFACE
The AMFData interface is a base interface for the AMBuffer and AMFSurface
interfaces. It is responsible for storing common properties of the video or audio data
objects (presentation time, duration, type, etc). The AMFData interface is also
responsible for memory type storage and memory conversion:
class AMFData : virtual public AMFPropertyStorage
{
public:
virtual AMF_MEMORY_TYPE GetMemoryType()=0;
virtual AMF_RESULT Duplicate(AMF_MEMORY_TYPE eMemType,AMFData
**ppData)=0;
virtual AMF_ERROR Convert(AMF_MEMORY_TYPE eMemType)=0;
virtual AMF_DATA_TYPE GetDataType()=0;
virtual void SetPts(amf_pts pts)=0;
virtual amf_pts GetPts()=0;
virtual void SetDuration(amf_pts duration)=0;
virtual amf_pts GetDuration()=0;
};

AMF ARCHITECTURE – BUFFER INTERFACE
The AMFBuffer interface provides access to an unordered memory buffer.
class AMFBuffer : virtual public AMFData
{
public:
virtual amf_size GetSize()=0;
virtual void* GetNative()=0;
virtual void AddObserver( AMFBufferObserver* pObserver) = 0;
virtual void RemoveObserver( AMFBufferObserver* pObserver) = 0;
};

AMF ARCHITECTURE – SURFACE INTERFACE
The AMFSurface interface represents a video frame and consists of a collection of
planes. It also holds the type of the frame: progressive, interlaced, stereoscopic, etc.
class AMFSurface : virtual public AMFData
{
public:
virtual AMF_SURFACE_FORMAT GetFormat() = 0;
virtual amf_size GetPlanesCount() = 0;
virtual AMFPlane* GetPlaneAt(amf_size index) = 0;
virtual AMFPlane* GetPlane(AMF_PLANE_TYPE type) = 0;
virtual AMF_FRAME_TYPE GetFrameType() = 0;
virtual void SetFrameType(AMF_FRAME_TYPE type) = 0;
virtual void AddObserver(AMFSurfaceObserver* pObserver) = 0;
virtual void RemoveObserver(AMFSurfaceObserver* pObserver) = 0;
};

AMF ARCHITECTURE – PLANE INTERFACE
The AMFPlane interface represents a single two-dimensional plane of a multi-plane
image format or the whole image for packed image format
class AMFPlane : virtual public AMFInterface
{
public:
virtual AMF_PLANE_TYPE GetType() = 0;
virtual void* GetNative() = 0;
virtual amf_int32 GetPixelSizeInBytes() = 0;
virtual amf_int32 GetOffsetX() = 0;
virtual amf_int32 GetOffsetY() = 0;
virtual amf_int32 GetWidth() = 0;
virtual amf_int32 GetHeight() = 0;
virtual amf_int32 GetHPitch() = 0;
virtual amf_int32 GetVPitch() = 0;
};

AMF ARCHITECTURE – CONTEXT INTERFACE - DIRECTX
The AMFContext interface is responsible for initializing and storing device-specific
data.
class AMFContext : virtual public AMFPropertyStorage
{
virtual AMF_RESULT AMF_STD_CALL Terminate() = 0;
virtual AMF_RESULT InitDX9(void* pDX9Device) = 0;
virtual void* GetDX9Device(AMF_DX_VERSION dxVersionRequired =
AMF_DX9) = 0;
virtual AMF_RESULT LockDX9() = 0;
virtual AMF_RESULT UnlockDX9() = 0;
virtual AMF_RESULT InitDX11(void* pDX11Device, AMF_DX_VERSION
dxVersionRequired = AMF_DX11_0) = 0;
virtual void* GetDX11Device(AMF_DX_VERSION dxVersionRequired =
AMF_DX11_0) = 0;
virtual AMF_RESULT LockDX11() = 0;
virtual AMF_RESULT UnlockDX11() = 0;

AMF ARCHITECTURE – CONTEXT INTERFACE – OCL, OGL
Context interface is responsible for initializing and storing API-specific data (OGL
devices, OCL contexts).
virtual AMF_RESULT InitOpenCL(void* pCommandQueue = NULL) = 0;
virtual void* GetOpenCLContext() = 0;
virtual void* GetOpenCLCommandQueue() = 0;
virtual void* GetOpenCLDeviceID() = 0;
virtual AMF_RESULT LockOpenCL() = 0;
virtual AMF_RESULT UnlockOpenCL() = 0;
virtual AMF_RESULT InitOpenGL(amf_handle hOpenGLContext, amf_handle
hWindow, amf_handle hDC) = 0;
virtual amf_handle GetOpenGLContext() = 0;
virtual amf_handle GetOpenGLDrawable() = 0;
virtual AMF_RESULT LockOpenGL() = 0;
virtual AMF_RESULT UnlockOpenGL() = 0;

AMF ARCHITECTURE – CONTEXT INTERFACE - ALLOCATION
 The AMFContext interface allows to allocate buffers and surfaces of a particular
memory type. The released objects are internally cached for future reuse.
virtual AMF_RESULT AllocBuffer(AMF_MEMORY_TYPE type, amf_size size,
AMFBuffer** ppBuffer) = 0;
virtual AMF_RESULT AllocSurface(AMF_MEMORY_TYPE type, AMF_SURFACE_FORMAT
format, amf_int32 width, amf_int32 height, AMFSurface** ppSurface) = 0;

AMF ARCHITECTURE – CONTEXT INTERFACE - WRAPPING
 The AMFContext interface allows to temporarily wrap an external memory
object(DX, OpenCL, OpenGL or Host memory) into an AMF surface or buffer object.
AMF allows to set an optional callback to track the release of AMF objects.
virtual AMF_RESULT CreateBufferFromHostNative(void* pHostBuffer,
amf_size size, AMFBuffer** ppBuffer, AMFBufferObserver* pObserver) = 0;
virtual AMF_RESULT CreateSurfaceFromHostNative(AMF_SURFACE_FORMAT
format,amf_int32 width, amf_int32 height, amf_int32 hPitch, amf_int32
vPitch, void* pData, AMFSurface** ppSurface, AMFSurfaceObserver*
pObserver) = 0;
virtual AMF_RESULT CreateSurfaceFromDX9Native(void* pDX9Surface,
AMFSurface** ppSurface, AMFSurfaceObserver* pObserver) = 0;
virtual AMF_RESULT CreateSurfaceFromDX11Native(void* pDX11Surface,
AMFSurface** ppSurface, AMFSurfaceObserver* pObserver) = 0;
virtual AMF_RESULT CreateSurfaceFromOpenGLNative( AMF_SURFACE_FORMAT
format, amf_handle hGLTextureID, AMFSurface** ppSurface,
AMFSurfaceObserver* pObserver) = 0;
};

AMF ARCHITECTURE – COMPONENT INTERFACE
Each component implements a common AMFComponent interface. The AMFComponent interface is
derived from AMFPropertyStorageEx and exposes a number of properties for the application to
discover and manipulate.
class AMFComponent : virtual public AMFPropertyStorageEx
{
public:
virtual AMF_RESULT Init(AMF_SURFACE_FORMAT format,amf_int32
width,amf_int32 height) = 0;
virtual AMF_RESULT ReInit(amf_int32 width,amf_int32 height) = 0;
virtual AMF_RESULT Terminate() = 0;
virtual AMF_RESULT Drain() = 0;
virtual AMF_RESULT Flush() = 0;
virtual AMF_RESULT SubmitInput(AMFData* pData) = 0;
virtual AMF_RESULT QueryOutput(AMFData** ppData) = 0;
virtual AMFContext* GetContext() = 0;
virtual AMF_RESULT SetOutputDataAllocatorCB(AMFDataAllocatorCB* callback)
= 0;
};

AMF TYPICAL WORKFLOW
Create
context
Create DX11
Device
Set device
to context
Create
Component
Set Properties
on component
Initiate
component
Allocate
input buffer
Fill
input buffer
Submit
Input
Query
Output
Drain
Query
Output
Terminate
Component
Terminate
Context

Init
Thread 1
AMF ONE-THREAD MODEL
Is EOF?
No
Prepare Input
object
Submit input
Drain
Terminate
Query Output
Have Data?
Process Data
No
Is EOF?
Start
End
Yes
Yes
Yes No

ENCODER SAMPLE - STATIC PARAMETERS
#include <d3d9.h>
#include <d3d11.h>
#include "../../../include/components/VideoEncoderVCE.h"
static amf::AMF_MEMORY_TYPE memoryTypeIn = amf::AMF_MEMORY_DX9;
static amf::AMF_SURFACE_FORMAT formatIn = amf::AMF_SURFACE_NV12;
static amf_int32 widthIn = 1920;
static amf_int32 heightIn = 1080;
static amf_int32 frameRateIn = 30;
static amf_int64 bitRateIn = 1000000L; // in bits, 1MB
static amf_int32 rectSize = 50;
static amf_int32 frameCount = 100;
static wchar_t *fileNameOut = L"./output.h264";
static bool PollAndWrite(amf::AMFComponent *encoder, FILE *f, bool
bDrain);
static void FillSurface(amf::AMFContext *context, amf::AMFSurface
*surface);
Init

ENCODER SAMPLE - INITIALIZATION
int _tmain(int argc, _TCHAR* argv[])
{
AMF_RESULT res = AMF_OK; // error checking can be added later
FILE *f = _wfopen(fileNameOut, L"wb"); // open output file
// initialize AMF
amf::AMFContextPtr context;
res = AMFCreateContext(&context);
if(memoryTypeIn == amf::AMF_MEMORY_DX9)
{
res = context->InitDX9(NULL); // can be DX9 or DX9Ex device
}
if(memoryTypeIn == amf::AMF_MEMORY_DX11)
{
res = context->InitDX11(NULL); // can be DX11 device
}
Init

ENCODER SAMPLE - ENCODER
amf::AMFComponentPtr encoder;
res = AMFCreateComponent(context, AMFVideoEncoderVCE_AVC, &encoder);
res = encoder->SetProperty(AMF_VIDEO_ENCODER_USAGE,
AMF_VIDEO_ENCODER_USAGE_TRANSCONDING);
res = encoder->SetProperty(AMF_VIDEO_ENCODER_TARGET_BITRATE, bitRateIn);
res = encoder->SetProperty(AMF_VIDEO_ENCODER_FRAMESIZE,
::AMFConstructSize(widthIn, heightIn));
res = encoder->SetProperty(AMF_VIDEO_ENCODER_FRAMERATE,
::AMFConstructRate(frameRateIn, 1));
res = encoder->Init(formatIn, widthIn, heightIn);
amf::AMFSurfacePtr surfaceIn;
// create input surface
res = context->AllocSurface(memoryTypeIn, formatIn, widthIn, heightIn,
&surfaceIn);
Init

ENCODER SAMPLE - MAIN LOOP
bool bFillSurface = true;
amf_int32 submitted = 0;
while(submitted < frameCount){
if(bFillSurface){
res = surfaceIn->Convert(memoryTypeIn);
FillSurface(context, surfaceIn);
}
res = encoder->SubmitInput(surfaceIn); // encode
if(res == AMF_INPUT_FULL){ // handle full queue
bFillSurface = false;
amf_sleep(1); // input queue is full: wait, poll and submit again
}else{
bFillSurface = true;
submitted++;
}
PollAndWrite(encoder, f, false); // poll and write all ready output
}
Prepare Input
object
Submit input

ENCODER SAMPLE - FILL SURFACE
static void FillSurface(amf::AMFContext *context, amf::AMFSurface *surface)
{
HRESULT hr = S_OK;
// no reference counting - do not Release()
IDirect3DDevice9 *deviceDX9 = (IDirect3DDevice9 *)context->GetDX9Device();
IDirect3DSurface9 *surfaceDX9=
(IDirect3DSurface9*)surface->GetPlaneAt(0)->GetNative();
hr = deviceDX9->ColorFill(surfaceDX9, NULL, D3DCOLOR_XYUV (128, 0, 128));
}
Prepare Input
object

ENCODER SAMPLE - POLLING
bool PollAndWrite(amf::AMFComponent *encoder, FILE *f, bool bDrain)
{
while(true){ // get all ready buffers
amf::AMFDataPtr data;
res = encoder->QueryOutput(&data);
if(res == AMF_EOF){return false;} // Drain complete
if(data != NULL){
amf::AMFBufferPtr buffer(data);
fwrite(buffer->GetNative(), 1, buffer->GetSize(), f);
}else{
if(bDrain){ amf_sleep(1);}
else { break; }
}
}
return true;
}
Query Output
Process Data

ENCODE SAMPLE - DRAIN & TERMINATE
while(true){// Drain
res = encoder->Drain();
if(res != AMF_INPUT_FULL){ // handle full queue
break;
}
amf_sleep(1); // input queue is full: wait and try again
PollAndWrite(encoder, f, false);
}
PollAndWrite(encoder, f, true); // poll and write the rest of output
surfaceIn = NULL;
encoder->Terminate();
encoder = NULL;
context->Terminate();
context = NULL;
fclose(f);
return 0;
}
Drain
Terminate

AMF TWO-THREAD MODEL
Is EOF?
Thread 2
Polling
Thread 1
Submit
Prepare Input
object
Submit input
Drain
Terminate
Init
Query Output
Have Data?
Process Data
Is EOF?
Thread Join
Thread 2 Start
Yes
Yes
No
No
Yes No

Q&A, REFERENCES
 AMD Media SDK website for downloads:
http://developer.amd.com/tools-and-sdks/media-sdk
 Latest driver download:
http://support.amd.com/en-us/download
 Media SDK Forum:
http://devgurus.amd.com/community/media-sdk
 Speaker: Mikhail Mironov, AMD Media SDK Software Architect:
mikhail.mironov@amd.com
 Developer Central Webinar Replays:
https://www.youtube.com/user/AMDDevCentral
 AMD Developer Central on Twitter (@AMDDevCentral):
https://twitter.com/AMDDevCentral
 Subscribe to be notified of future Webinars:
http://developer.amd.com/community/newsletter/subscribe/

DISCLAIMER
The information contained herein is for informational purposes only, and is subject to change without
notice. While every precaution has been taken in the preparation of this document, it may contain
technical inaccuracies, omissions and typographical errors, and AMD is under no obligation to update
or otherwise correct this information. Advanced Micro Devices, Inc. makes no representations or
warranties with respect to the accuracy or completeness of the contents of this document, and
assumes no liability of any kind, including the implied warranties of noninfringement, merchantability
or fitness for particular purposes, with respect to the operation or use of AMD hardware, software or
other products described herein. No license, including implied or arising by estoppel, to any intellectual
property rights is granted by this document. Terms and limitations applicable to the purchase or use of
AMD’s products are as set forth in a signed agreement between the parties or in AMD's Standard
Terms and Conditions of Sale.
AMD, the AMD Arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc.
Other product names used in this publication are for identification purposes only and may be
trademarks of their respective companies.
USE OF THIS PRODUCT IN ANY MANNER THAT COMPLIES WITH THE MPEG ACTUAL OR DE FACTO VIDEO AND/OR AUDIO STANDARDS IS EXPRESSLY
PROHIBITED WITHOUT ALL NECESSARY LICENSES UNDER APPLICABLE PATENTS. SUCH LICENSES MAY BE ACQUIRED FROM VARIOUS THIRD PARTIES
INCLUDING, BUT NOT LIMITED TO, IN THE MPEG PATENT PORTFOLIO, WHICH LICENSE IS AVAILABLE FROM MPEG LA, L.L.C., 6312 S. FIDDLERS GREEN
CIRCLE, SUITE 400E, GREENWOOD VILLAGE, COLORADO 80111.
Windows® is a registered trademark of Microsoft Corporation. DirectX ® is a registered trademark of Microsoft Corporation. OpenCL™ is a
trademark of Apple Inc. used by permission by Khronos. Other names are for informational purposes only and may be trademarks of their
respective owners.
© 2014 Advanced Micro Devices, Inc. All rights reserved.

Media SDK Webinar 2014

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (15)

Similar to Media SDK Webinar 2014

Similar to Media SDK Webinar 2014 (20)

More from AMD Developer Central

More from AMD Developer Central (8)

Recently uploaded

Recently uploaded (20)

Media SDK Webinar 2014