Wednesday, August 23, 2017

RawCopy sending output on TCP securely

Fixed a few bugs in the TcpSend option of RawCopy, and thought it would be a good opportunity to explain how it can be used to send output securely.

The implementation of the TcpSend option is very basic. It merely open a tcp socket and send the data on the wire to the destination. Obviously this is not secure because the data is sent unencrypted in plaintext. Now there are a few options available for using that feature in a secure way. The following examples are all based on the assumption that you have a ssh server endpoint available. This way we can transfer the files securely through a SSH tunnel.



In all examples we use local port 6666 and remote port 7777 at the tunnel endpoints. That is, we use port 6666 on the RawCopy part and port 7777 on the remote netcat (or similar) part. For RawCopy we will thus use a command like this;


The remote netcat command would then be something like this;
nc -l -p 7777 >logfile.bin

For the creation of the encrypted tunnel:

Putty

Configure a regular ssh connection and specify local and remote destination in the tunnel options like this;

Now there's not yet any listening end, so open netcat inside the putty shell to receive the file from rawcopy;


Plink

This is the command line equivalent of putty that. Logic is still the same. Create the tunnel by plink command like this;


And use netcat in the shell as with putty.

OpenSSH

We will use the ssh client of Microsoft's Win32-OpenSSH, which is a fork of OpenSSH. This time we will nest all commands together in one command so that it could be suited for scripting. In short we will add the remote netcat command into the ssh command like this;


That would be much better with automation.

Saturday, August 19, 2017

Detection of filesystem patterns in encrypted containers

As part of the latest journey into VeraCrypt, I noticed a few characteristics of the containers with modified volumes. The tests performed here are done with VeraCrypt 1.22 on Windows using default settings, but applies equally well to Linux. In order to spot these charateristics, you need to have at least 2 different samples of the container to analyze. If you don't have more than 1 sample, then you might not be able to observe these characteristics. Of cource, if you are provided with a password to the container you can normally just decrypt it and get the data. But you may not have access to any hidden volume there, and might still be wondering if a hidden volume could be present. By using this method, you may be able to say, even without the decryption key, with a little more confidence that;
  • It is likely that a hidden volume can be present in that container.
  • It is not likely that a hidden volume can be found in that container.
  • This chunk of random looking data might be an encrypted container with a filesystem such as NTFS or FAT.
  • Provide a point in time when the content within the encrypted container was modified, even though timestamps within the encrypted volume has been tampered with.
  • The mode of operation used in the encrypting process seems to be XTS and not CBC (or vice versa).
After the last round of playing with bitmaps in http://plainbinary.blogspot.com/2017/08/more-playing-with-veracrypt-and-hybrid.html it quickly became evident that a visual representation of the data in a bmp could become handy. Some more playing and looking at diffs of containers before and after content was modified, it suddenly got a whole lot more interesting. At this point it is appropriate to mention that I started searching for previous research on the topic, and found a few references. First and foremost, the documentation itself, has a warning. See the first bulletpoint at https://www.veracrypt.fr/en/Security%20Requirements%20for%20Hidden%20Volumes.html containing a brief section explanaining the security issue with exposing more than 1 sample of the container from different points in time. And some more significant references relating to this ;

Find a second version of container

Finding a second version may not be easy or possible at all. But some places they can be found in are;
  • Backups or older copies.
  • Left over like in unallocated on the filesystem.
  • Volume shadow copies.

Create the image

For the post I prepared a tool called MakeImage that will transform any piece of binary data into a bmp that can be visually inspected in a proper graphics viewer. Grab the tool at https://github.com/jschicht/MakeImage . The tool will let you specify the width of image. It is hardcoded to use ARGB 32 bits per pixel in the bmp. What we will do is create a bitmap representing the disk, and we need our bitmap to display sector size in width. Since each pixel is 4 bytes, the width must be 512/4=128 pixels (X). Now we have 1 sector from disk per line in the image.  For the bytes to display properly in the image, the bmp is constructed such that the lines over Y are flipped upside down, whereas the the bytes per line on X are preserved in original order. The resulting image may thus be quite large (in heigth), and not all programs will display it properly. This is an example of what we will be dealing with;

That sample is just 4 mb in size and well suited for visual inspection. Not much can be deduced from that file when looking at it with no earlier version to compare with. It just looks like random data.

Produce a diff

Now we will decrypt the volume and store a file in there. Unmount the volume and let MakeImage create another bitmap that we can work on. Then let us compare the images and produce a diff. A very nice and easy and free tool for this diffing task is the compare component from ImageMagick. No need to reinvent the wheel here. Download the package from https://www.imagemagick.org/download/binaries/ImageMagick-7.0.6-7-portable-Q16-x64.zip and use a command like this;

compare image1.bmp image2.bmp diff.bmp

The resulting output file diff.bmp will have flashing red pixels for those that differ over the 2 images.

Identification of filesystem pattern

This is in itself not overly impressive, but merely a nice way of letting us visually analyze the difference. If we were to look at a bunch of numbers, it is sometimes hard to see properly what is there. Especially when dealing with a diff of encrypted bytes, where we don't have concrete things like size and timestamp to relate bytes to. Instead, we are left with the possibility to spot a vague pattern. The identification of such pattern may be programatically solved, but will not be covered here, at least this time. So what kind of pattern is there to spot? Let's first go through briefly how we can produce a diff. Since our pixels are built of 4 bytes we can't be more granular in the visual inspection than saying that 1 or all of these 4 bytes has changed. Turns out that VeraCrypt in the default setting will encrypt blocks of 0x10 bytes. That makes perfect sense as the tool reports a block size of 128 bits;


Which of course is expected; https://www.veracrypt.fr/en/Encryption%20Algorithms.html Therefore our 4 bytes per pixel restriction is not the determining factor for how specific we can be. Said differently, when looking at the diff of the encrypted container, we will see blocks of 4 red pixels where data has changed. If there was a need to be more granular with the diff, we could have used 1 byte per pixel, and used 2048 on width Y.

FAT

We will first investigate both the encrypted diff and then the unencrypted diff, and observe the similarity. This cropped image is from the encrypted diff taken before and after a file was copied into the FAT volume;


The first chunk of red pixels is seen at Y pixel 258 (pixels start at 0). That would mean sector number 259. Remember the VeraCrypt specification that says the standard header is from offset 0 - 0x10000 and the header for the hidden volume from offset 0x10000 - 0x20000. So the 2 headers are located on sector 1 - 256, which means that for the unencrypted volume the first 2 sectors are untouched, and the first modification is within the third sector. Let us take a closer look at the image above. Here is a crop from Y pixel 256 (and zoomed in 400%), which should be the start of the unencrypted volume;


The next thing I did was to produce dd images of both the unencrypted volumes, produce bitmaps of them and then generate a diff. This is the diff of the same region, but cropped from Y pixel 0, since there is no VeraCrypt header in the decrypted volumes;


We can immediately spot the clear similarity. Now let's take a look at what changed on the FAT volume specifically. Short version is that the first 2 chunks are for FAT 1 and FAT 2, while the third is for Root directory. It seems the location of these 3 pieces as well as the blocks between them, will highly depend on the size of the volume, cluster size and FAT version. It may not be possible to conclude on volume size by just looking at the diff. But we may possibly identify FAT 1 and FAT 2 based on their identical change, FAT 1 being close to volume start and their distance apart (in sectors) being in the multiple of 2. Here's some text applied to the encrypted diff.


Even though changes on FAT may not be very clear in pattern, it could still be determined based on elimination. 

Physical location of standard vs hidden volume

Now let's take a step back and create a container with both standard and hidden volume. Same size of 4 mb for the standard (outer) volume and 500 kb for the hidden volume within the standard volume. Then put a file into both the volumes and made a diff. This image is scaled down so it fits within a reasonable window, and servers to illustrate the relative location of the standard vs hidden volume within the container;


No need to show the diff for the standard volume as it is almost identical to the diffs presented above. We will instead look at the diff for the hidden volume. Here's a crop of the hidden part zoomed in;


It clearly has some similarity to the standard volume observation. The main difference is that the hidden volume changes were found towards the end of the container. That's just how it needs to be as it resides within unallocated randomized data, and the further out in the file the hidden volume is placed, the less likely it is to be overwritten. Based on the pattern found and the fact that the changes were found towards end of container, it is likely a hidden volume. If the standard volume was unlocked and the location of changes are mapped to unallocated space within the standard volume, it is very likely the hidden volume.

Identification of pattern for NTFS

The FAT filesystem is less complex than NTFS, and will contain much less activity and diff changes than for NTFS. It might therefore be easier to spot an NTFS pattern than a FAT pattern. But that's up to the judging eye. I have myself more experience with NTFS analysis and will try to show some clear pattern that can be explained. The images are generated from volumes that have either a single file copied or a single file modified. But even then, the chunks of red are significant. There are more changes to NTFS than has been provided in the below images. Take a look at this cropped image part, which is zoomed in 500 %;


As you can see there are some patterns to spot. Some of these are very specific to NTFS. Here's some text applied to the same image part. Let me try to explain the image.


First a brief explanation of the abbreviations;
  • LSN: The $LogFile sequence number in MFT record header. It reflects the last FS transactions that conserned a given file. 
  • USA: Update sequence array. Is the array of bytes that is checked and replace during fixups. Heavily used.
  • SI-TS: $STANDARD_INFORMATION timestamps. These timestamps are 4 x 8 bytes. That is why the red chunk is wider than the others. Few of these timestamps are updated regularly.
  • USN: Update sequence number found in $STANDARD_INFORMATION. Used with UsnJrnl. Think of it as LSN with $LogFile. Updated regularly when UsnJrnl is active.
  • FIXUPS: The 2 bytes at end of sector that are replaced with the value found in USA. Heavily used.
The strongest indicator is probably the column of red on the right side. This is the fixups, that NTFS uses for internal integrity checks in various metafiles including MFT records and INDX's. This 2 byte field is located at the end of each sector. For every remount of the volume these are regenerated, and is why we see this so clearly.

INDX

The first chunk is an INDX. We can deduce that since the height is 8 pixels, and INDX's are 8 sectors in size (4096 bytes) with fixups on each sector. The USA is normally located at offset 0x28 in INDX. That explains why we have 8 unmodified pixels (32 bytes) before it.

$MFT

The presence of $MFT is also fairly easy to spot. The MFT record has a somewhat fixed structure. Of course not all attributes are constant, but the header is always present while $STANDARD_INFORMATION is almost always present. The fixups are also always present. MFT records can be of size 1024 or 4096, though 1024 is the most usual seen. For 1024 byte records it means every second sector will most of the time have little changes. This is because attributes tend to fit within the first 512 bytes of the record for most regular files. This pattern is also very clear, with changes on every second line.

$LogFile

This file is also fairly easy to spot as it is heavily used on Windows.


The RSTR section of the $LogFile will not have much modifications except some values in the header and the restart area as well as the fixups. The RSTR is 2 times 0x1000 bytes, followed by a bunch of RCRD's. The RCRD content is updated frequently which is why you see all the red. All RCRD's have fixups that you can vaguely spot in the large red section.

$TxfLog.blf

Turned out this one had an easy to spot pattern worth mentioning as well;


This feature is rarely used by applications so I would assume that pattern to stay like that. However, if transaction were used, then it would look different. Though, it would still contain fixups, so maybe more in pattern like $LogFile but with less red. It can also be turned completely off during format, in which case it would not be present.

Password changes

What if we changed the password for the encrypted volumes? Turns out the byte changes are reflected identically for both standard and hidden volumes. There will be a complete red line like this;


So the complete sector changed. For a given volume type we will see 2 such lines, though at different offsets.
For the standard volume the offsets (Y pixels) are;
  • 0x0 (Y = 0)
  • EOF - 0x20000 (max of Y - 128)
For the hidden volme the offsets are;
  • 0x10000 (Y = 128)
  • EOF - 0x10000 (max of Y - 256)

Other container types

At this point I got curious what other type of containers would leave for diff. The only other encryption solutions I tried for the test was BestCrypt, ProxyCrypt and Bitlocker. All of them (going with default settings) showed similar behaviour to VeraCrypt on the diffs, with good patterns to spot. So, in the end nothing revolutionary, just block ciphers doing encryption on fixed sized blocks.

Changing Mode of Operation to CBC

This post is already way too long, but I'll throw in a brief comparison of XTS mode (which is covered in above examples) vs CBC mode. In theory it should distroy some of the pattern as the chaining process causes any block to be XOR'ed by the previous blocks ciphertext. The CBC test was done with BestCrypt. Surprisingly the NTFS pattern is still identifyable, but in a less distinct way. Here is how $TxfLog.blf now looks like;



As can be seen, it is still familiar. Now, take a look at this image covering an INDX and $MFT;

Still, the pattern is somewhat visible. Interestingly this could mean that when analyzing such diffs, it could be possible to identify which mode of operations that has been used in the encryption process, at least if you were to choose one of XTS and CBC. I could be off track with this one, but I thought it was an interesting enough observation to share anyway.

Sunday, August 13, 2017

More playing with VeraCrypt and hybrid files

I thought I should expand a bit on the VeraCrypt game started in the previous post http://plainbinary.blogspot.no/2017/07/steganography-with-modified-veracrypt.html

This time I'll focus on the legacy version though. We will investigate what the encrypted container bytes may;

  • Look like in a bmp image.
  • Sound like in wav track.
  • Look like when turned into a raw video format like yuv.
The three examples we will go through are similar to the procedure described in example 4 of the previous blogpost referenced at the top. That is with a method like http://keyj.emphy.de/real-steganography-with-truecrypt/ where the container is a hybrid file that fulfill the structure of another file format and still works as a VeraCrypt container that can be mounted. The hybrid files can thus be managed in multiple ways.

For the examples I whipped together som code to prepare such containers. They can be found in the usual place at https://github.com/jschicht/MakeContainer Look out for those with names having -Legacy appended.

1. Encrypted bytes as a bmp image.


First create the container in VeraCrypt and use hidden volume feature, because we can't really use the standard/normal volume when used in a bmp. The reason for that is because the header is overwritten with the BMP header, and the last few bytes in the file may be bytes filled for alignment purpose. The way VeraCrypt works is that when the original header is invalidated for some reason, there is an option to load backup headers (both standard and hidden) from the last 0x20000 bytes of the file. Now in the case of the bmp trick, both places are tampered with, leaving us with the option to load hidden volume because its header (offset 0x10000 - 0x20000 from start) is still intact. Strictly speaking, it is possible to load the backup headers if you have a tool available that can load the encrypted bytes from a defined offset range and present it like a disk. This is what I explained in previous post when using the aim_ll tool from Arsenal Image Mounter. But forget about that for the moment, as we'll do this exploration from a different angle this time. Next download the tool at https://github.com/jschicht/MakeContainer/tree/master/MakeContainer-Bmp-Legacy and turn the container into a bmp. This is how such an image looks like;


It may come as no surprise that there is no obvious pattern to spot. That is of course an important point with encryption, to produce bytes that can't be distinguished from random bytes. I could mention that the tool will let you specify the pixel width, while the height is automatically created based on the chosen width. The reason for the appended bytes is because a pixel line (width) can't be broken or partial. That is, the smaller width, the less bytes you will possibly have to append. The header will look like a regular bmp header like this;


The encrypted bytes start at offset 0x36. It is worth noting that the way bmp works, is that the original (pixel) bytes are flipped 180 degrees on the X, and then turned upside down. When creating the special containers we obviously can't do that because it would break the VeraCrypt format. We thus need to make the bytes on disk as they are expected by VeraCrypt, but the bytes are distorted visually as interpreted and displayed through the bmp (inverted). But that does not really matter for the view right now as random data will still look like random data on the pixel side regardless of whether it is flipped 180 degress by X or turned upside down.

Mounting the encrypted container fairly easy. Just load the bmp into VeraCrypt and provide the password for the hidden volume.

2. Encrypted bytes as sound in a wav.


Create the container in VeraCrypt. You can use both standard and hidden volume with this file format. Similarly as with bmp, parts of the header is overwritten. Here's what the header may look like;


Fortunately with the wav format, there is no need to align data size. We can therefore use the backup headers for the standard volume. Usage of the hidden volume is just like normal. Go to https://github.com/jschicht/MakeContainer/tree/master/MakeContainer-Wav-Legacy and download the tool. There's an option to configure the number of channels. The tool will then output a ready made and valid wav to be played. At this point you might be wondering what it really sounds like. Here's a sample; https://drive.google.com/file/d/0B1M9FFDeCvMpUVVzOXdZRnRtUmc/view?usp=sharing The password of the standard volume is "joakim". It plays fine in Windows Media Player;


and also in VLC;


In order to mount the standard volume, we must use the backup headers. Configure the mount options like this;


The interesting thing with wav is that you can burn your "sound" compilation onto a CD, and still mount the encrypted container directly from the CD. For that we would need to also turn on the read-only mount option;


As already mentioned you can use both standard and hidden volumes, and since I used the backup headers in the above screenshot, it's because I was mounting the standard volume. As seen in the next screenshot, the sound file plays fine and VeraCrypt can mount the volume too;



3. Encrypted bytes as a raw video file.


First prepare a volume. With this method we also have some restrictions to handle. Due to the way raw video files and yuv in particular are built, is that there is no real header as with most other file formats. You simply need to know what frame size and type the file is expected to be in, and then just pass the parameter to the player. To strictly adhere to the specific encoder it is supposed to be in, the file may need some bytes appended. For this reason, you can use both standard and hidden volumes, in the normal way without using backup headers. In fact, the backup headers may not work at all because of the reason just explained. Get this tool https://github.com/jschicht/MakeContainer/tree/master/MakeContainer-RawVideo-Legacy for the preparation of the container. There you will have the option to specify the XY dimension of the video, and the tool will align file size according to the chosen XY and the rgba or yuva444p format. The output filename will contain the details to play it properly. In order to play the yuv file, there are a few options. For instance;

#1
vlc --demux rawvideo --rawvid-fps 25 --rawvid-width 320 --rawvid-height 240 --rawvid-chroma I420 file_320x240_rgba.yuv

#2
ffplay -f rawvideo -pixel_format rgba -video_size 320x240 -framerate 25 file_320x240_rgba.yuv

#3

When playing with raw video it quickly became evident that these players actually will attempt playing anything you feed it. The video will mostly look like with the bmp sample;



The concept of data hiding is far from new. A few related links worth mentioning;
Lastly I will stress the point that even though the encrypted container is masked within another file format, it is relatively easy to spot that there's something strange about it when you investigate the file.