Requirements
   
Prerequisite knowledge
Some basic familiarity with Flash Media Server administration or
application development.
Required Products
Flash Media Server (Download Trial)
 
Administrators
You should have installed FMS and opened the configuration
files at least once, and have an understanding of the needs of your installation.
Additional requirements
Flash Media Authentication Add-in (optional)
User level
Advanced

Application developers 
You should have a basic grasp of writing server-sideActionScript and the information on whether or not you may insertyour own Application.xml configuration override for your application.
Required Products
Flash Media Server (Download Trial)
 
Applying the simple statement, "less is more," to your Flash Media Server (FMS) installation can mean fewer headaches, security nightmares, or problems from customers. Your FMS installation is up and running; you want to keep things that way.
In this article you will learn some of the ways to prevent your FMS from being used for unintended purposes, as well as understand how to disable features you don't need and gain insights about some things you may not have known about your server.
 
Why harden Flash Media Server?
Flash Media Server is, and has always been, a product that was determined to make everything easy and to run right out of the box. This has been successful, but we've led ourselves to a place where FMS being a fully functional Swiss Army knife is causing a problem.
The problem is one of users being unaware of the functionality running on their FMS installations and of their choices to disable, limit, or control that functionality as needed. This is a security concern, as pirating live streams, having authenticated content, and generous protocol configurations allow for all kinds of security abuses that users should be made aware of.
You can protect your FMS installation by properly configuring it against security abuses. The term for this procedure in the industry is hardening, and this guide will show you what you can do to harden your FMS installation.
Server administrator guidelines
The following hardening guidelines are for the FMS administrator. This guide assumes that the administrator has root access to this individual system and FMS as a whole. The sections below will be broken up into:
Items in redshould be highlighted as the most significant portions, to be used for getting the most critical hardening done now.
 
System-level configurations
Hardening an FMS installation requires being aware of the system on which FMS runs. In particular, it's important to mind excessively permissive network settings and access controls, as detailed below.
 
Network
A firewall in front of your FMS deployment can prevent unwanted network access on uncontrolled software. Please consider firewalling ports not in use for FMS and whitelisting the ports you need. By default, FMS uses ports 1935, 443, and 80 to receive client communications. Many configurations don't need to make use of port 443 at all and can consider blocking that port on the firewall. Most installations will want access to 1935 and 80 for RTMP and RTMPT respectively.
RTMFP uses the UDP protocol to communicate. Many firewalls block UDP outright on all ports; consider your needs for opening port ranges for proper RTMFP communication (details should be found in the RTMFP docs).
FMSAdmin cannot be accessed securely (RTMPS), and thus Adobe currently advises blocking port 1111 to prevent external access to the FMSAdmin service. Access to the FMSAdmin console will be consequently restricted to within your firewalled network.
 
System access
Make sure that only Administrators or trusted users have access to FMS installation folder, configurations, and in the case of Windows, registry keys. Uncontrolled access to these files can allow a user to take over, modify, or otherwise compromise your FMS installation.
Access to FMS media content should be limited on a requirement basis, where only FMS and necessary content uploaders and downloaders may access it. Uncontrolled access to these areas could compromise FMS performance and endanger customer content.
On Unix, FMS should be configured to run as a user with limited permissions per the UID and GID settings in Server.xml. This FMS user, as recommended, should have read-only access to configuration files to prevent unwanted changes in the event of a security breach or accidental overwrite. The FMS user should also have only read and execute permissions to FMS executables, preventing coordinated attacks against the FMS DLLs. Finally, the FMS user should have limited to no permissions outside its own folder and content directories.
Care should be taken, when configuring FMS to run as an alternate user, that FMS has the necessary permissions to read all contents of its own folder, plus read/write access for the FMS_INSTALL_DIR/tmp directory and any locations where streams may be recorded. FMS also needs the ability to launch interprocess communications—such as to access shared memory and create mutexes. Finally, many users need FMS to communicate with network shares and present network login credentials for a particular user to authenticate network sources. Given these details, you should show caution in changing FMS user permissions and test thoroughly such configuration changes.
If possible, the FMS logging and recording locations should be located off the system disk location to prevent accidental writing from occupying too much disk space.
Configuration of FMS system limits in areas such as file handles, semaphores, sockets, and other limited system resources should take into account maximum load needs, but not to unlimited levels. Settings for these resources are highly installation-dependent. Establishing appropriate settings for each of these system resources should be determined by monitoring them while the system is in normal operation, adding on appropriate additional capacity for high loads, and finally limiting total usage to prevent overuse.
 
Applications
Sample applications deployed with FMS permit live streaming and recording by default. Once your deployment is prepared and not using the VOD and live applications, consider removing them to no longer allow the permissive and well-known sample applications.
 
Server.xml configurations
Configure the following elements in the FMS Server.xml file to enhance your installation's security:
 
Root/Server/NetworkingIPV6:
If an IPV6 network is not deployed for your FMS configuration, you may consider disabling this approach to your FMS.
 
Root/Server/CrossDomainPath: 
FMS can be used to provide a crossdomain.xml file to Flash Players or browsers as required. This can enhance the security model of your deployment by restricting the domains from which your FMS will accept cross-domain connections. For more information, please see the Adobe Cross Domain Policy File Specification.
 
Root/Server/SSL: 
Configuring your server to speak RTMPS with the included SSL configurations grants the proven security of OpenSSL to client-server communications. Certificate locations and configurations must be provided here to effectively communicate via RTMPS.
 
Root/Server/Process:
As mentioned in System Access, on Linux platforms, create an FMS user with carefully chosen permissions as needed. Set FMS to run as this user with the Process element.
 
Root/Server/Mask:
When FMS creates files, set the file-writing mask on Linux platforms to control the read/write access of FMS files written, so that only necessary users may access this file. Note that FMS itself must have access to write files it creates without error.
 
Root/Server/AdminServer/RTMP/RTMPE:
If you must communicate with the FMSAdmin server externally (not recommended), do employ the RTMPE protocol. This offers some protection over RTMP basic, but as RTMPE is a versioned protocol without certificates, that protection is not complete. FMSAdmin, as noted above, should be firewalled to allow access only from trusted sources.
 
Root/Server/AdminServer/Process:
Do set the FMSAdmin server to run as the FMS user, thereby limiting the reach of any accidental API usage or exploit.
 
Root/Server/AdminServer/(Allow|Deny|Order):
To further constrict the FMSAdmin interface: if you can, build a whitelist of acceptable IPs from which to administer the FMSAdmin. Note: This is not a substitute for firewalling away FMSAdmin access.
 
Root/Server/AutoCloseIdleClients:
Removing idle clients from your FMS installation can help keep system resource levels to a minimum and avoid idle client–based attacks or simple waste.
Root/Server/ResourceLimits/FLVCache/MaxSize: On a VOD deployment, this will be the largest use of memory on your FMS system. The setting for this should be balanced between having a deep and efficient cache and potentially overwhelming system resources. Consider, for instance, the number of FMSCore processes that may be active at once with a full cache. Typically, the following rules of thumb apply:
 
  • Scope: The scope (as determined in Application.xml; see that section of this hardening guide) will determine how many FMSCore processes FMS launches, depending on traffic to your various adaptors, vhosts, applications, and instances. Consider the most cores this may create at once.
  • Distribute: Now consider if FMSCore processes will be further split into multiple instances. This multiplies the previous number.
  • Rollover: Further, if rollover is active and the MaxCores element is set greater than 1, that number multiplies again the total number of FMSCore processes possible. Note that this is deactivated by default.
 
Finally now, consider, with n FMSCores consuming m bytes, the system resources available. Make sure this number does not consume available memory on your FMS system to prevent potential memory exhaustion due to overuse and swapping.
 
Root/Server/ResourceLimits/Protocol/RTMP/Edge/MaxConnectionRate:
Consider setting a maximum connection rate to prevent a rapid incoming wave of users from overwhelming your system resources to process it. FMS should be capable of handling hundreds to thousands of connections per second, but script and plug-in processing will slow this operation, so it's important this this setting be measured before being enacted to make sure that the chosen level is not hindering server performance, or is set too high to prevent damage.
 
Root/Server/SWFVerification:
Use this element to control the location of master SWF files that will pass SWF Verification for all applications requiring it. This provides a master key to SWF Verification requirement. Be sure that individual clients aren't uploading their application's SWF files here, as this will allow those SWF files to connect to ALL applications.
 
Root/Server/Https
If you don't plan on using the built-in Apache server for delivering HTTP downloads, make sure that this configuration is disabled.
 
Users.xml
As I noted previously, it's advised that the FMSAdmin not be exposed to the general public, as it offers only basic cleartext password security and is not suitable for public exposure. Configure the following elements of User.xml to further harden your FMS installation:
 
Root/UserList/User/Password
It's advised that you activate Password encryption to prevent mistaken config file access from compromising passwords.
 
Root/UserList/User/[Allow/Deny]
Consider only allowing IPs that are known to be safe to connect to the FMSAdmin. This is not a substitute, but a supplement to firewall restrictions for the FMSAdmin.
 
Root/AdminServer/HTTPCommands/Allow:
Allowing only the admin commands that are necessary will prevent powerful but unnecessary commands from being exploited.
 
Adaptor.xml
Configure the following elements of Adaptor.xml:
Adaptor/Allow: If the target customer IPs for connecting to FMS are known in advance, consider locking down those connections via adding them to the Allow/Deny elements here.
 
Adaptor/HTTPTunnel/Enable: If the RTMPT and RTMPTE protocols are unneeded, you should deactivate them here.
 
Adaptor/SSL: Pay careful attention to the SSL settings used for activating RTMPS and getting the proven security of SSL for your RTMP. Particular attention should be paid to the SSLCertificateFile, SSLCertificateKeyFile, and CipherSuite settings.
 
Adaptor/HTTPIdent: Enabling and providing information within HTTPIdent is a necessary part of some player/server infrastructures, but if unnecessary, it can be providing FMS version information and presence info to an attacker, so consider carefully the information provided within HTTPIdent and if it's needed in your deployment.
 
Adaptor/HTTPUserInfo: The same guidance as HTTPIdent applies to HTTPUserInfo: deactivation is the preference if unneeded.
 
VirtualHost level configurations
Virtual Hosts represent FMS for one or more domains. It's below the networking and system access level, but there are more particular FMS considerations such as limitations to FMS resources, and application controls to take into account.
 
VHost.xml
Configure the following elements of VHost.xml:
 
VirtualHost/AutoCloseIdleClients: Activating this configuration can prevent waste, but can also prevent attacks involving large numbers of idle-but-present clients. Using this will make sure someone cannot slowly consume all the connections to your FMS.
 
VirtualHost/ResourceLimits: Each of these resource limits should be set to a reasonable level to prevent users or attackers from overwhelming your FMS resources with too many connections, streams, or shared objects.
 
VirtualHost/VirtualDirectory: Make sure to root your virtual directories safely so that there are no sensitive materials below your virtual directory root. For example, on Windows, using C:\ as your
VirtualDirectory root for Streams would grant read access to all files below that level with the proper play attempts.
 
VirtualHost/Allow: Consider allowing only specific domains to connect to your FMS if that information is known in advance.
 
VirtualHost/Proxy/SSL: Use these configurations to control outgoing edge/origin communications to use RTMPS and ensure a high level of security.
 
Application.xml
The Application.xml file in the conf folder is the default Application.xml file, and users may, in their Application folders, define overrides of certain settings on this default if so allowed. You should prevent users from overriding values that they should not edit. To do this, specify override="no" as an attribute to any XML element in the Application.xml file. If override="no" is specified and a node has children, those children will inherit the override setting. Therefore, for maximum security, add override="no"
to the Application element at the root of Application.xml and no changes can be made.

Application/Process/Scope and Application/Process/Distribute: Carefully consider the use of these two elements, as they determine the number of FMSCore processes running without a rollover configuration. Scope determines the grouping of logical FMS operations into executables. Distribute further spreads the number of core processes. Be careful not to specify too many executables—for instance, changing the Scope to inst and creating 1000 app instances is a recipe for overwhelming system resources with 1000 FMSCore processes. Further distribution will again multiply the number of FMSCores. Determine the number of desired FMSCores running on a system first. Adobe's recommendation is no more than 16 and an advised target of only 3 per virtual host, as configured by default.
 
Application/Process/Lifetime: Don't rollover FMSCores unless advised to by an Adobe support staff member to avoid an issue being investigated. Setting this can further multiply the number of FMSCores running on a system and have unintended side effects.
 
Application/AuthCloseIdleClients:
As in VHost.xml, closing idle clients can reduce your system drain and prevent attacks with too many idle clients.
 
Application/JSEngine/RuntimeSize:
Do restrict the total size of JavaScript memory to a level where garbage collection can effectively prevent users from unintentionally draining system resources.
 
Application/JSEngine/MaxTimeout:
This can prevent JavaScript infinite loops from stealing system performance. It should be noted that it costs some JavaScript performance to check for infinite loops, so choose as needed.
 
Application/JSEngine/FileObject: Adobe strongly recommends to NOT set a VirtualDirectory for the FileObject. This configuration can be used by script to write to the chosen location and would give a potential attacker a desired target to control. Unless clearly needed, do not enable a FileObject VirtualDirectory, or if so, make sure that it is carefully chosen to be a safe sandbox location.
 
Application/StreamManager/StorageDir:
Setting a StorageDir element changes the sandbox path for writing out recorded files or reading VOD ("video on demand") files. Take care when setting this to move the file sandbox to a safe location.
 
Application/Recording/*:
All the max size elements for recording may be set to values that can prevent a rogue recorder on FMS from consuming too many system resources. If you can plan for a maximum recording size, then use this attribute to enforce that.
 
Application/SharedObjManager/Max*:
The two max elements, MaxProperties and MaxPropertySize, on SharedObjManager can prevent a SharedObject from getting too large and consuming too many resources. If you know in advance that you won't be using shared objects or that your size will be limited, please make use of this configuration.
 
Application/Client/Bandwidth:
These settings provide the default bandwidth constraint settings for a stream on this application. The BandwidthCap limits the maximum bandwidth that can be set in script. These values should be carefully observed, as they can both save from a bandwidth hog calculation, but if set too low, they can cause buffering problems if invoked on a large client stream by mistake. Choose these values carefully.
 
Application/BandwidthDetection:
Think carefully about whether clients should be exposed to BandwidthDetection. It is a helpful metric to have for pacing a client connection; however, it can also be used to drain server resources for connections as it has some associated cost per connection. If the number of connections in use will be high, consider curtailing use of this or whether it will be needed.
 
Application/Access/*SampleAccess:
These two configurations, AudioSampleAccess and VideoSampleAccess, control whether or not connection Flash Players may take audio or video samples from the RTMP content being streamed. It's important to consider the rights holders when setting these configurations and whether those streaming should be able to remove pieces of the content and store it offline.
 
Application/Debug:
Debug connections are only intended for development purposes. On all shipping products please set the AllowDebugDefault to false (default) and the MaxPendingDebugConnections to 0 to disable this feature.
 
Application/SWFVerification:
SWF Verification is a tool to ensure that only SWF files of your choosing can connect to a specific application. It's advised to activate this feature and ensure that rogue SWF files may not connect and exploit the resources of your application.
 
Application/SWFVerification/UserAgentExceptions:
If needed, open a UserAgentException for your encoder of choice to SWFVerification. You should also cover this hole with the FMLE Authentication Plugin to complete your security model. The combination of Access module controlled authentication for encoders (such as the FMLE Plugin) and SWF Verification for standard players is a recommended simple path to authentication. Please see the comments around UserAgentExceptions in the FMS Configuration files for examples of how to configure for the UserAgent for your encoder.
 
Application/DisallowedProtocols:
If you want to be most secure, consider disallowing protocols that are not secure. For instance, listing RTMP, RTMPT here will ensure that only RTMPE and RTMPS variants will be able to connect to FMS, thereby forcing encrypted connections to your application. In addition, make sure that your space here lists any versions of protocols that may be considered insecure now; for instance, RTMPE1.

Developer guidelines

The following guidelines are for an application developer on FMS. Sections below will include modifying your Application.xml file, writing your server-side ActionScript, securing your communication with FMS, and performing authentication.
Items in red should be highlighted as the most significant portions, to be used for getting the most critical hardening done now.
 
Application.xml settings
As a developer, you have the same configurations available to you in Application.xml as server administrators do. See the previous section on this topic for Application.xml hardening guidelines.
 
Server scripting
The following are recommendations for writing your server script in a secure fashion.
application.allowDebug: Setting allowDebug to false makes doubly sure that your intention for this application is that no debug style connections be allowed. Debug connections can be a useful tool for entering applications through a back door; but should be shutdown in production.
 
client.agent:
UserAgent
strings can be used to identify particular types of RTMP-speaking clients connecting. In addition, it can be used to determine versions and platform information. If only certain connection types are expected—for instance, only Flash Players and not encoders of a particular type—then checking this can be a useful defense tool. Additionally, this is personal information for connecting clients and should be considered private informational data of your application and not shared with the general public.
 
client.ip:
One of the most powerful pieces of information is the connecting IP address of the client. This value cannot be reset or modified by FMS and represents the client IP address directly or the most recent proxy exposed for this client, thus allowing tracing your connection back to its previous link. This information can be used to screen clients as desired; for instance, if certain IPs are expected and others aren't or have been banned. Additionally, this is sensitive information about your user base, so it should be restricted information from the public at large.
client.readAccess and writeAccess: Use these configurations to restrict clients from reading or writing media on FMS. This is one of your best restrictions of permission and can be easily enforced. Clients that don't need
writeAccess
should have this set to the empty string to completely deny write access and enforce a higher level of security. This access pertains to all FMS stored and ephemeral media, including live and recorded streams and shared objects.
 
client.audioSampleAccess and videoSampleAccess:
Set this to allow/deny clients from sampling portions of streams that they download. The default behavior is to not allow sampling, but in some cases that ability is desired. Think carefully on the exposure to capturing client content that enabling this allows.
 
client.referrer or pageUrl:
These fields indicate the loaded URI a SWF file experiences—this allows to make sure that the RTMP connection is coming from a desired source. This verification can be tampered with so it's not recommended over SWF Verification—but is still a good check that the chosen URI for your SWF file is in use.
 
client.nearNonce and farNonce:
These fields are the nonce values used in negotiation of the RTMP handshake with FMS. As desired, they can be used to provide a random one-time data source for authentication or other necessary such protocols.
file object: It's critical to lock down all users of the FMS file object. This is a powerful API giving its controller the ability to write/read files from the FMS sandboxed locations. When used in conjunction with virtual directories, this can be leveraged to write files outside of the FMS application sandbox and can give its controller a potential path to exploit. It is highly recommended to make limited or no use of the File API for users and make no use of the FMS File Virtual Directories to remove the sandbox restriction.
 
loadVars object:
Direct access to
loadVars
should not be made for connecting clients. This is because FMS would then become a proxy for client requests to be made outward, and these requests may execute in a context more privileged than the connecting client. For instance, if FMS is exposed to connections coming from outside the firewall, but has back-end permission within a firewall one would want to contain access to these APIs to make sure that only desired requests are made.
 
Stream.setVirtualPath:
This command changes the virtual path used by streams and can be used to divert plays and recordings. Using this command will have significant effects on what and where items can be played from and put to, so one would not want to give this command's parameters to external user control.
 
Secure communication
Communicating securely with clients is the most effective way to make sure that your FMS session isn't tampered with or intercepted.
RTMPS communication leverages the proven security of SSL to wrap your RTMP session. Employing RTMPS complete with certificates and the appropriate ciphers will allow for maximum security. Do test for the appropriate amount of additional load imposed by RTMPS usage and consider the use of SSL accelerators.
RTMPE-based communication offers some of the benefits of RTMPS, but not all. It trades performance and certificate-less communication for being a versioned protocol under private Adobe control. This means that you may make use of RTMPE for security, but you should make sure that you're using the most recent versions of RTMPE only, and only relying on RTMPE where the extra performance tradeoff is appreciated over the proven security of RTMPS.
 
Authentication
User authentication relies upon secure communication for a reliable foundation. Wherever possible leverage the best available Secure Communication above to prevent even the best authentication techniques from being targeted for interception.
The use of SWF Verification will allow FMS to restrict connection to a particular application from only the desired SWF file. This will ensure that rogue SWF files or non–Flash Player agents are unable to connect to your application. If some connecting agents are incapable of SWF Verification, then UserAgentExceptions will have to be added to the SWF Verification configuration.
Authentication of non-SWF players, for example an FMLE video encoder may take place through the FMLE Authentication Addin.
Clients may implement their own authentication system via client and server script, or using the access/authorization plug-ins based upon user connection information—for example, requiring a separate authentication token to be provided at connect time and retrieving that token via script or adaptor can enable/disable privileges and provide a high granularity and control solution.
Where to go from here
Sign up for the Security Notification Service from Adobe. This is a valuable service notifying you of critical exploit fixes.
For more questions about FMS security or hardening your FMS installation, post questions to the Flash Media Server forums.
The FMS Support and FMS Escalations teams will make every effort to answer questions and post for the benefit of all. In some security-sensitive cases, Adobe technical staff may ask to take discussions offline to handle issues appropriately until they can be resolved safely for the public. If you're already aware of a security issue involving FMS or another Adobe product, please report it directly to Adobe using the security response page.
For more information on the Adobe Product Security Incidence Response Team, please consider following their blog.