The security considerations for file upload

The overall goal is to build a set of defensive layers that tightly control
the process of uploading the file and later retrieval of the file. The user will
always interact indirectly with the file and never directly access the file
system without application control.
This document is constructed into 2 parts, A section mentioned about
unrestricted file upload vulnerability, the one B is describe some of methods
to implement for details. Read it carefully to use them most effectively.
A- Unrestricted File Upload Vulnerability.
1. Description
Uploaded files represent a significant risk to applications. The first step in many
attacks is to get some code to the system to be attacked. Then the attack only needs to
find a way to get the code executed. Using a file upload helps the attacker accomplish
the first step.
The consequences of unrestricted file upload can vary, including complete system
takeover, an overloaded file system or database, forwarding attacks to back-end
systems, and simple defacement. It depends on what the application does with the
uploaded file and especially where it is stored.
There are really two classes of problems here. The first is with the file metadata,
like the path and file name. These are generally provided by the transport, such as
HTTP multi-part encoding. This data may trick the application into overwriting a
critical file or storing the file in a bad location. You must validate the metadata
extremely carefully before using it.
The other class of problem is with the file size or content. The range of problems
here depends entirely on what the file is used for. See the examples below for some
ideas about how files might be misused. To protect against this type of attack, you
should analyze everything your application does with files and think carefully about
what processing and interpreters are involved.
2. Risk Factors
- The impact of this vulnerability is high, supposed code can be executed in the
server context or on the client. The likelihood of a detection for the attacker is
high. The prevalence is common. As a result the severity of this type of
vulnerability is high.
- The web server can be compromised by uploading and executing a web-shell
which can run commands, browse system files, browse local resources, attack
other servers, and exploit the local vulnerabilities, and so forth. This may also
result in a defacement.
- An attacker might be able to put a phishing page into the website.
- An attacker might be able to put stored XSS into the website.
- This vulnerability can make the website vulnerable to some other types of attacks
such as XSS.
- Picture uploads may trigger vulnerabilities in broken picture libraries on a client
(libtiff, IE had problems in the past) if the picture is published 1:1.
- Script code or other code may be embedded in the uploaded file, which gets
executed if the picture is published 1:1.
- Local vulnerabilities of real-time monitoring tools, such as an antivirus, can be
exploited.
- A malicious file (Unix shell script, windows virus, reverse shell) can be uploaded
on the server in order to execute code by an administrator or webmaster later -- on
the server or on a client of the admin or webmaster.
- The web server might be used as a server in order to host of malware, illegal
software, porn, and other objects.
3. Examples
Attacks on application platform
- Upload .jsp file into web tree - jsp code executed as web user
- Upload .gif to be resized - image library flaw exploited
- Upload huge files - file space denial of service
- Upload file using malicious path or name - overwrite critical file
- Upload file containing personal data - other users access it
- Upload file containing "tags" - tags get executed as part of being "included" in a
web page
Attacks on other systems
- Upload .exe file into web tree - victims download trojaned executable
- Upload virus infected file - victims' machines infected
- Upload .html file containing script - victim experiences Cross-site Scripting
(XSS)
4. Weak Protection Methods and Methods of Bypassing

Using Black-List for Files Extensions

Some web applications still use only a black-list of extensions to prevent from
uploading a malicious file.
- It is possible to bypass this protection by using some extensions which are
executable on the server but are not mentioned in the list. (Example: file.php5,
file.shtml, file.asa, or file.cer)
- Sometimes it is possible to bypass this protection by changing some letters of
extension to the capital form (example: file.aSp or file.PHp3).
- Using trailing spaces and/or dots at the end of the filename can sometimes cause
bypassing the protection. These spaces and/or dots at the end of the filename will
be removed when the file wants to be saved on the hard disk automatically. The
filename can be sent to the server by using a local proxy or using a simple script
(example: file.asp ... ... . . .. .., file.asp , or file.asp.).
- A web-server may use the first extension after the first dot (.) in the file name or
use a specific priority algorithm to detect the file extension. Therefore, protection
can be bypassed by uploading a file with two extensions after the dot character.
The first one is forbidden, and the second one is permitted (example:
file.php.jpg).
- In case of using IIS6 (or prior versions), it might be possible to bypass this
protection by adding a semi-colon after the forbidden extension and before the
permitted extension (example: file.asp;.jpg).
- In case of using IIS6 (or prior versions), it might be possible to bypass this
protection by putting an executive file such as ASP with another extension in a
folder which ends with an executive extension such as .asp (example:
folder.asp\file.txt). Besides, it is possible to create a directory just by using a
file uploader and ADS (Alternate Data Stream). In this method, filename should
end with ::$Index_Allocation or :$I30:$Index_Allocation to create a directory
instead of a file (example: newfolder.asp::$Index_Allocation creates
newfolder.asp as a new directory).
- This protection can be completely bypassed by using the e.g. control characters
like Null (0x00) after the forbidden extension and before the permitted one. In this
method, during the saving process all the strings after the Null character will be
discarded. Putting a Null character in the filename can be simply done by using a
local proxy or by using a script (example: file.asp%00.jpg). Besides, it would
be perfect if the Null character is inserted directly by using the Hex view option
of a local proxy such as Burpsuite or Webscarab in the right place (without
using %).
- It is also possible to create a file with a forbidden extension by using NTFS
alternate data stream (ADS). In this case, a : sign will be inserted after the
forbidden extension and before the permitted one. As a result, an empty file with
the forbidden extension will be created on the server (example: file.asp:.jpg).
Attacker can try to edit this file later to execute his/her malicious codes. However,
an empty file is not always good for an attacker. So, there is an invented method
by the author of this paper in which an attacker can upload a non-empty shell file
by using the ADS. In this method, a forbidden file can be uploaded by using this
pattern: file.asp::$data..
- In Windows Servers, it is possible to replace the files by using their short-name
(8.3). (example: web.config can be replaced by uploading web~1.con)
- Sometimes combination of the above can lead to bypassing the protections.
Using White-List for Files Extensions
Many web applications use a white-list to accept the files extensions. Although
using white-list is one of the recommendations, it is not enough on its own. Without
having input validation, there is still a chance for an attacker to bypass the protections.
- the 3rd, 4th, 5th, and 6th methods of last section apply here as well.
- The list of permitted extensions should be reviewed as it can contain malicious
extension as well. For instance, in case of having .shtml in the list, the
application can be vulnerable to SSI attacks.

Using Content-Type from the Header

Content-Type entity in the header of the request indicates the Internet media
type of the message content . Sometimes web applications use this parameter in order
to recognize a file as a good one. For instance, they only accept the files with the
Content-Type of text/plain.
- It is possible to bypass this protection by changing this parameter in the request
header by using a local proxy.

Using a File Type Recognizer

Sometimes web applications intentionally or unintentionally use some functions (or
APIs) to check the type of the file in order to do further process. For instance, in case
of having image resizing, it is probable to have image type recognizer.
- Sometimes the recognizers just read the few first characters (or header) of the
files in order to check them. In this case, an attacker can insert the malicious code
after some valid header.
- There are always some places in the structure of the files which are for the
comments section and have no effect on the main file. And, an attacker can insert
malicious codes in these points.
- Also, it is not impossible to think about a file modifier (for example an image
resizer) which produces malicious codes itself in case of receiving special input.
5. Prevention Methods (Solutions to be more secure)

Some special recommendations for the developers and webmasters:

- Never accept a filename and its extension directly without having a white-list
filter.
- It is necessary to have a list of only permitted extensions on the web application.
And, file extension can be selected from the list. For instance, it can be a select
case syntax (in case of having VBScript) to choose the file extension in regard to
the real file extension.
- All the control characters and Unicode ones should be removed from the
filenames and their extensions without any exception. Also, the special characters
such as ;, :, >, <, / ,\, additional ., *, %, $, and so on should
be discarded as well. If it is applicable and there is no need to have Unicode
characters, it is highly recommended to only accept Alpha-Numeric characters
and only 1 dot as an input for the file name and the extension; in which the file
name and also the extension should not be empty at all (regular expression: [a-zA-
Z0-9]{1,200}\.[a-zA-Z0-9]{1,10}).
- Limit the filename length. For instance, the maximum length of the name of a file
plus its extension should be less than 255 characters (without any directory) in an
NTFS partition.
- It is recommended to use an algorithm to determine the filenames. For instance, a
filename can be a MD5 hash of the name of file plus the date of the day.
- Uploaded directory should not have any execute permission.
- Limit the file size to a maximum value in order to prevent denial of service
attacks (on file space or other web applications functions such as the image
resizer).
- Restrict small size files as they can lead to denial of service attacks. So, the
minimum size of files should be considered.
- Use Cross Site Request Forgery protection methods.
- Prevent from overwriting a file in case of having the same hash for both.
- Use a virus scanner on the server (if it is applicable). Or, if the contents of files
are not confidential, a free virus scanner website can be used. In this case, file
should be stored with a random name and without any extension on the server
first, and after the virus checking (uploading to a free virus scanner website and
getting back the result), it can be renamed to its specific name and extension.
- Try to use POST method instead of PUT (or GET!)
- Log users activities. However, the logging mechanism should be secured against
log forgery and code injection itself.
- In case of having compressed file extract functions, contents of the compressed
file should be checked one by one as a new file.


B- Guideline for implement:
Upload all files to a file-share and not directly to database as a blob:
If the files are uploaded to the database as a blob then the data that resides
on the file will not be validated thus may lead to injection attacks. For example an
attacker may use this vulnerability to inject malicious data in to the database and
use it later to perform other attacks. So, it is always a better way to upload the
files to a file-share instead of the database as a blob.
In general, try to store client-supplied files on their own partition. Avoid
storing them on the same partition as the web site, and always avoid storing
them on the system drive:
- An attacker might be able to exhaust free space on the system drive or the web
site drive, resulting in unexpected behavior by the OS or the web site.
Alternately, if the attacker can do a directory-traversal attack, then the attacker
might be able to replace system or web site files, thereby compromising the
web server system.
- Allowing a user to upload the files either to a system drive or website partition
may lead to DOS attacks.
If client-supplied files are stored on the web site partition, make sure that
they never reside within the web site directory tree:
Allowing client-supplied files in the web site directory tree might result in
an attacker being able to upload a malicious file, and then navigate to it using an
HTTP GET request. This would cause the server to execute the attacker-supplied
page using the identity of the web application account.
Alternately, an attacker might upload a file containing malicious script, and then
lure a victim user to access the same file via GET, effectively resulting in a cross-
site script exploit.
The HTTP POST method is preferable to the HTTP PUT method:
When using the PUT method, you need to take extra care to ensure that
user-supplied files cannot be accessed with other methods such as GET. This
would result in the same cross-site script attack described above.
Scan client-supplied files for viruses using a well - known industry standard
virus scanner:
Allowing client-supplied files to be uploaded to a server without a virus
scan might result in an attacker being able to upload malicious files such as
viruses.
Make sure that the well - known industry standard virus scanner is used on
the server and the virus signatures are up to date.
Ensure all incoming/outgoing Files are scanned and they are scanned on real time
basis.
Never assume that files which pass a virus scan are completely safe, they
may contain a malicious code that could help the attacker to perform other attacks
like code injection.
Check and restrict the size of uploaded files if possible, and allocate server-
side resources sparingly.
- Attackers who can upload extremely large files can potentially deplete or
exhaust server resources, resulting in unexpected behavior by the server or
web site.
- Ensure application checks for the length so that it doesnt potentially deplete
or exhaust server resources.
When storing client-supplied files on disk, generate your own file paths and
file names. Do not allow user-supplied data to influence the path or name of
the file as it is actually stored on disk:
- By depriving attackers of the ability to influence how your application
performs file-level I/O, you all but eliminate a number of possible attacks,
such as directory traversal, induced filename collisions, and more.
- If it is necessary to store the file on disk using a user-supplied name, you
should do the following:
a. Use the Path.GetFilename() method to safely retrieve a valid filename from
the client-supplied value.
b. Use a directory path you choose yourself, without allowing user input to
influence the directory path.
Lock down access to specific users and security-groups requiring access to
the upload share.
Treat client-supplied files like any other element of user input and validate,
validate, validate.
- Verify that the file is of the type the user is permitted to upload.
- Discard any file which does not have an extension or name that is explicitly
allowed or expected by your application.
- Also consider inspecting the internal contents of the file to ensure that it is of
the type advertised by the user-supplied name or extension.
Finally: Remember the #1 rule of good web application security, see it below. All users are
evil!
The following basis rules to Secure File Uploads, Control File Upload are introduced
below, based summarization all of principle to secure file upload by users:
1. Create a new file name
Do not use the user supplied file name as a file name on your local system. Instead,
create your own unpredictable file name. Something like a hash (md5/sha1) works as it is
easily validated (it is just a hex number). Maybe add a serial number or a time stamp to
avoid accidental collisions. You may add a secret to the name to make it harder to guess
the file name. If you need to keep the original file name: use a look-up table to link the
validated user supplied file name to the server created name.
2. Store the file outside of your document root

If your document root is /var/www/html, create a directory /var/www/uploads and use it
to store uploaded files. That way, an attacker will not be able to retrieve the file directly.
This will allow you to provide fine grained access control. The file will not be parsed by
the server's application language module but the source of the file will be streamed. Also,
we should check the directory property is not execute , forbid any execute files from
users.

3. Check the file size

You should set a maximum file size in the upload form, but remember: It is just advisory.
Make sure to check the file size after the upload completed. Be in particular careful if you
allow the upload of compressed files and later uncompress them on the server. This
scenario is very hard to secure.

4. Extensions are meaningless

However, even Apache doesn't always behave the way you expect it to. Try
'something.php.x' in Apache and chances are that php code will be executed. Its a feature.
If you stream a file back to the user, the extension isn't what matters, but the Content-
Type header and the file's header. It is best to use the "file" command on unix to check
the file type. But even this is not fool proof. It will just check the first few bytes. In PHP
for example, a file may start with a GIF header, but later if the PHP engine sees a
"<?php" tag, it will happily interpret an embedded PHP script.

5. Try a malware scan

The extension is right, and you checked that the file is actually a valid JPEG file per it's
header. However, it could still be a malicious JPEG using one of the many image parser
bugs to exploit clients downloading the file. There is no great defense against this as far
as I am aware. One possible work around is to "rebuild" the file. Convert the JPEG to a
GIF and back to a JPEG. This will likely strip out any malicious feature. But this
technique could expose your servers to just the same image parser bugs.

Simultaneously, buy many licenses to use Malwarebytes Anti-Exploit
(https://www.malwarebytes.org/antiexploit/ ) to stop unknown threats to Windows, which
is a well - known industry standard malware/virus scanner, to Shield media players
(including Microsoft Windows Media Player, VideoLAN VLC Player, QuickTime
Player, Winamp Player) while they open media files. It need be installed into servers, be
mandatory.
6. Keep tight control of permissions

Any uploaded file will be owned by the web server. But it only needs read/write
permission, not execute permissions. After the file is downloaded, you could apply
additional restrictions if this is appropriate. Sometimes it can be helpful to remove the
execute permission from directories to prevent the server from enumerating files.

7. Authenticate file uploads

File uploads, in particular if these files are viewable by others without moderator review,
have to be authenticated. This way it is at least possible to track who uploaded an
objectionable file.

8. Check magic number of file type.

Reference: http://www.garykessler.net/library/file_sigs.html
http://en.wikipedia.org/wiki/Magic_number_(programming)


9. Limit the number of uploaded files

Many developers limit the file size, but not all limit the number of files uploaded in a
request. Make sure to apply reasonable limits. But be also ready for a DoS attack that just
uploads a large number of small files. Pick an appropriate directory structure to limit the
number of files per directory and pick an appropriate file system.