1. Using Files

Written by Fuad Kamal

There are many ways to store data in an Android app. One of the most basic ways is to use files. Similar to other platforms, Android uses a disk-based file system that you can leverage programmatically using the File API. In this chapter, you’ll learn how to use files to persist and retrieve information.

Reading and writing files in Android

Android separates the available storage into two parts, internal and external. These names reflect an earlier time when most devices offered built-in, non-volatile memory (internal storage) and/or a removable storage option like a Micro SD card (external storage). Nowadays, many devices split the built-in, permanent storage into separate partitions, one for internal and one for external. It should be noted that external storage does not indicate or guarantee that the storage is removable.

There are a few functional differences between internal and external storage:

• Availability: Internal storage is always available. On the other hand, external storage is not. Some devices will let you mount external storage using a USB connection or other options; this generally makes it removable.
• Accessibility: By default, files stored using the internal storage are only accessible by the app that stored them. Files stored on the external storage system are usually accessible by everything. However, you can make files private.
• Uninstall Behavior: After uninstalling the app, files saved to the internal storage are removed and the ones in the external storage are not. The exception to this rule is if the files are saved in the directory obtained by getExternalFilesDir.

Here’s a usage hint: Use internal storage when you don’t want the user or any other apps to access the files, like important user documents and preferences. Use external storage when you want to allow users or other apps access to the files. This might include images you capture within the app.

Getting started

To get started, you’ll build an app that uses internal storage. Locate using-files/projects and open starter using Android Studio. Sync the project and run the app on a device or emulator. For now, you can ignore the warnings in the code.

The Simple Note app has a simple user interface with two EditTexts, one for the filename and one for a note. There are also three buttons along the bottom: READ, WRITE and DELETE. The user interface looks like this:

The sample for this project includes three sub-packages:

• model: Includes a simple data class to represent a single note in Note.kt. NoteRepository.kt contains the interface declaration with methods to add, get and delete a Note. ExternalFileRepository, InternalFileRepository and EncryptedFileRepository are implementation classes of NoteRepository.
• ui: This package includes MainActivity.kt. MainActivity implements the OnClick methods for each button. Because the code to read, write, encrypt and delete is abstracted behind NoteRepository and placed into separate classes, the code to handle the button click events remains the same regardless of the type of storage being utilized. This code is dependent on a single repository, and only the concrete type of the repository needs to change.
• app: This package includes Utility.kt, which contains a utility function to produce Toast messages.

Using internal storage

The internal storage, by default, is private to your app. The system creates an internal storage directory for each app and names it with the app’s package name. When you uninstall the app, files saved in the internal storage directory are deleted. If you need files to persist — even after the app is uninstalled — use external storage.

Are you ready to see internal storage in action?

Writing to internal storage

Open MainActivity.kt. Notice the code immediately below the class definition:

private val repo: NoteRepository by lazy { InternalFileRepository(this) }

This is a lazy value. It represents an object of a class that implements NoteRepository. There’s a separate implementation for each storage type demonstrated in this chapter.

The repo is initialized the first time it’s used and will be utilized throughout MainActivity. This includes the button click events that call the add, get and delete methods required by NoteRepository.

In onCreate(), locate binding.btnWrite.setOnClickListener() add the following code for the WRITE button’s click event:

// 1
if (binding.edtFileName.text.isNotEmpty()) {
  // 2
  try {
    // 3
    repo.addNote(Note(binding.edtFileName.text.toString(),
        binding.edtNoteText.text.toString()))
  } catch (e: Exception) { // 4
    showToast("File Write Failed")
  }
  // 5
  binding.edtFileName.text.clear()
  binding.edtNoteText.text.clear()
} else { // 6
  showToast("Please provide a Filename")
}

Here’s how it works:

1. Use an if/else statement to ensure the user entered the required information.
2. Put repo.addNote() into a try/catch block. Writing a file can fail for different reasons like permissions or trying to use a disk with not enough space available. Using a try/catch block will ensure the app doesn’t crash.
3. Call addNote(), passing in a Note that contains the filename and text provided in the EditText fields.
4. If writing the file fails, display a Toast message and write the stack trace of the error to Logcat.
5. To prepare the interface for the next operation, clear the text from edtFileName and edtNoteText.
6. If the user didn’t enter a filename, display a toast message within the else block. showToast() is a utility function that exists in Utility.kt.

The code for READ and DELETE click events are similar to what you added for WRITE; these already exist in the sample project.

Now, open InternalFileRepository.kt and locate addNote(). Then, add the following to the body of the method:

context.openFileOutput(note.fileName, Context.MODE_PRIVATE).use { output ->
  output.write(note.noteText.toByteArray())
}

This code opens the file in fileOutputStream using the Context.MODE_PRIVATE flag; using this flag makes this file private to this app. The FileOutputStream is a Closeable resource so we can manage it using use(). The note’s text is converted to a ByteArray and written to the file.

Build and run. Enter Test.txt for the file name and some text for the note. Then, tap WRITE. If the write is successful, the EditText controls will clear. Otherwise, the stack trace is printed to Logcat.

Now that you’ve learned how to read, write and delete files from internal storage, wouldn’t it be nice to see a visual representation of the files in the file system?

Viewing the files in Device File Explorer

In Android Studio, there’s a handy tool named Device File Explorer. This tool allows you to view, copy and delete files that are created by your app. You can also use it to transfer files to and from a device.

Note: A lot of the data on a device isn’t visible unless the device is rooted. For example, in data/data/, entries corresponding to apps on the device that aren’t debuggable aren’t expandable in the Device File Explorer. Much of the data on an emulator isn’t visible unless it’s an emulator with a standard Android (AOSP) system image. Be sure to enable USB debugging on a connected device.

Open the Device File Explorer by clicking View ▸ Tool Windows ▸ Device File Explorer or by clicking the Device File Explorer tab in the window toolbar.

The Device File Explorer displays the files on your device. Open data > data > com.raywenderlich.android.simplenote > files; you’ll see Test.txt and any other files you’ve saved. Files are saved in a directory with the same name as the app’s package name.

Note: The file location depends on the device; some manufacturers tweak the file system, so your app directory might not be where you expect it. If that’s the case, you can locate the folder using the app’s package name as this never changes.

The files stored on the external storage are located in sdcard/Android/data/app_name/.

At the top of the Device File Explorer, there’s a drop-down you can use to select the device or emulator. After making your selection, the files appear in the main window. You can expand the directories by clicking the triangle to the left of the directory name.

Right-click the filename, and a menu pops up that allows you to perform different operations on the file.

1. Open lets you open the file in Android Studio.

2. Save As… lets you save the file to your file system.

3. Delete allows you to delete the file.

4. Synchronize synchronizes the file system if it’s changed since the last run of the app.

5. Copy Path copies the path of the file to the clipboard.

Now, it’s time to learn how to make your app read files.

Reading from internal storage

In InternalFileRepository.kt , replace return in getNote()with the current code:

// 1
val note = Note(fileName, "")
// 2
context.openFileInput(fileName).use { stream ->
  // 3
  val text = stream.bufferedReader().use {
    it.readText()
  }
  // 4
  note.noteText = text
}
// 5
return note

Here’s how it works:

1. Declare a Note, passing in a fileName and an empty string so that a valid object gets returned from this function even if the read operation fails.
2. Open and consume the FileInputStream with use().
3. Open a BufferedReader with use() so that you can efficiently read the file.
4. Assign the text that was read to the file to note.noteText.
5. Return note.

Build and run. Next, enter the name of a file you previously saved and then tap READ. The note’s text displays in the app.

And that’s it! Your app can now write and read notes. Up next, you’ll write the code to delete a file.

Deleting a file from internal storage

In InternalFileRepository.kt, replace return of deleteNote() with the following line of code:

return noteFile(fileName).delete()

This function returns a value of successful file deletion.

Build and run. Delete a file by tapping DELETE; you’ll see the appropriate message. To confirm the file was deleted, use the Device File Explorer.

Internal storage is great for storing private data in an app. But what if you want to store data temporarily? To do this, you can use Internal Cache Files.

Internal cache files

Each app has a special and private cache directory to store temporary files. Android may delete these files when the device is low on internal storage space, so it’s not safe to store anything other than temporary files in this space. There’s also no guarantee that Android will delete these files for you, so you must maintain this directory yourself.

To write to the internal cache directory, use createTempFile() as shown in the following example:

File.createTempFile(filename, null, context.cacheDir)

A good use case for temporary files is when you’re uploading images to a server. You may not need the image persisted on the device, but you still need to upload some files. You’d store the image in this temp file, upload it, and then delete the file upon completion.

Next, it’s time to look at how to store files on External Storage.

Using external storage

External storage is appropriate for data you want to make accessible to the user or other apps. Files saved on the external storage system aren’t deleted after uninstalling the app. The external storage is made up of standard public directories. Files saved to the external storage are world-readable and can be modified by enabling mass storage and transferring the files to the computer via USB.

External storage isn’t guaranteed to be accessible at all times; sometimes it exists on a physically removable SD card. Before attempting to access a file, you must check for the availability of the external storage directories, as well as the files. You can also store files in a location on the external storage system, where they will be deleted by the system when the user uninstalls the app.

Now that you know the theory, it’s time to replace usage of internal storage with external. You’ll start by adding the necessary permissions to the manifest.

Adding permissions in the manifest

To use external storage, you must first add the correct permission to the manifest. If you wish to only read external files, use the READ_EXTERNAL_FILE permission.

<uses-permission 
  android:name="android.permission.READ_EXTERNAL_STORAGE" />

If you want to both read and write, use the WRITE_EXTERNAL_STORAGE permission.

<uses-permission 
  android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

Beginning with API level 19, reading or writing files in your app’s private external storage directory doesn’t require the above permissions. If your app supports Android API level 18 or lower, and you’re saving data to the private external directory only, you should declare that the permission is requested only on the lower versions of Android by adding the maxSdkVersion attribute:

<uses-permission 
  android:name="android.permission.WRITE_EXTERNAL_STORAGE"
  android:maxSdkVersion="18" />

In this app, you’ll be reading and writing to the external storage, so add the WRITE_EXTERNAL_STORAGE permission inside the <manifest> element in AndroidManifest.xml.

Writing the notes to external storage

Now that the correct permissions are in place, it’s time to write the file to the external storage. Open ExternalFileRepository.kt and add the following code to addNote():

// 1
if (isExternalStorageWritable()) {
  // 2
  FileOutputStream(noteFile(note.fileName)).use { output ->
    // 3
    output.write(note.noteText.toByteArray())
  }
}

Here’s how it works:

1. Check to see if the external storage is available.
2. Open a FileOutputStream with use().
3. Write note.noteText to the file.

Next, in MainActivity.kt, change the instance of the NoteRepository you’re initializing to the following:

private val repo: NoteRepository by lazy { ExternalFileRepository(this) }

Finally, build and run to write a file to the external storage. Name the file as ExternalStorageTest.txt and add a random note. Then, press WRITE.

To view the file using the Device File Explorer, look in sdcard/Android/data, within the app’s package name folder.

Reading from external storage

To read from the storage, open ExternalFileRepository.kt, and replace return of getNote() with the following code:

val note = Note(fileName, "")
// 1
if (isExternalStorageReadable()) {
  // 2
  FileInputStream(noteFile(fileName)).use { stream ->
    // 3
    val text = stream.bufferedReader().use {
      it.readText()
    }
    // 4
    note.noteText = text
  }
}
// 5
return note

For the most part, the procedure here is the same as reading from the internal storage but with a small difference. Here’s how it works:

1. Ensure the external storage is readable.
2. Open and consume the FileInputStream, with use(), as you did before.
3. Open a BufferedReader with use so that you can efficiently read the file.
4. Assign the text that was read to the file to note.noteText .
5. Return the note.

The above code blocks rely on the following two functions: isExternalStorageWritable() and isExternalStorageReadable(). The first one determines if the external storage is mounted and ready for read/write operations, whereas the other determines only if the storage is ready for reading.

You’re ready to add the capability to delete a file from external storage.

Deleting a file from external storage

In ExternalFileRepository.kt, replace return with the following code into deleteNote():

return isExternalStorageWritable() && noteFile(fileName).delete()

The first part of the condition checks if the external storage can be written to or altered; the second part, if the first condition is true, returns the result of deleting the file. This way, you can be sure that the file will be deleted only if you can manipulate external storage.

Securing user data with a password

Security is important for the credibility of your app, especially when it comes to securing users’ private data. Storing data on external storage allows the data to be visible to other apps. That’s why it’s advised to avoid using external storage. Or at least doing so, without a strong security system and encryption. To prevent users from installing the app on external storage you can add android:installLocation="internalOnly" to the manifest file.

Another best practice you can use to enhance your app’s security is to prevent the contents of the app’s private data directory from being downloaded with adb backup. You do this by setting the android:allowBackup="false" in the manifest file.

One way to secure your data beyond the best practices listed above is to encrypt the files before writing them to the external file system with a user-provided password.

Using AES and Password-Based Key Derivation

The recommended standard to encrypt data with a given key is the AES (Advanced Encryption Standard). In this example, you’ll use the same key to encrypt and decrypt data - known as symmetric encryption. The preferred length of the key is 256 bits for sensitive data.

It’s not realistic to rely on the user to select a strong or unique password. That’s why it’s never recommended to use passwords directly to encrypt the data. Instead, produce a key based on the user’s password using Password-Based Key Derivation Function or PBKDF2.

PBKDF2 produces a key from a password by hashing it over many times with salt. This creates a key of a sufficient length and complexity, and the derived key will be unique even if two or more users in the system used the same password.

In this example, passwordString that represents the user’s password has been hardcoded at the top of EncryptedFileRepository.kt.

Find encrypt() and add the code inside the empty try block:

// 1
val random = SecureRandom()
// 2
val salt = ByteArray(256)
// 3
random.nextBytes(salt)

Here’s how it works:

1. Generate a random value using the SecureRandom class. This guarantees the output is difficult to predict as SecureRandom is a cryptographically strong random number generator.
2. Create a ByteArray of 256 bytes to store the salt.
3. Pass the salt to nextBytes() which will fill the array with 256 random bytes.

Next, add the following code below the previous code block to salt the password.

// 4
val passwordChar = passwordString.toCharArray() 
// 5
val pbKeySpec = PBEKeySpec(passwordChar, salt, 1324, 256) // 1324 iterations
// 6
val secretKeyFactory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1")
// 7
val keyBytes = secretKeyFactory.generateSecret(pbKeySpec).encoded
// 8
val keySpec = SecretKeySpec(keyBytes, "AES")

Here’s how it works:

4. Convert the password into a character array.
5. Pass the password in char[] form, along with the salt, to PBEKeySpec, as well as the number of iterations, 1324, and the size of the key, 256.
6. Generate an instance of a SecretKeyFactory using PBKDF2WtihHmacSHA1.
7. Pass the pbKeySpec to the secretKeyFactor.generateSecret() method and assign the encoded property which returns the key in bytes.
8. Finally, keySpec is produced to use when you initialize the Cipher.

All of these steps seem a bit complex, but it’s the kind of thing that’s always the same to use; only the data changes. You’re almost done!

Using an initialization vector

The recommended mode of encryption when using AES is the cipher block chaining, or CBC. This mode takes each next unencrypted block of data and uses the XOR operation with the previous encrypted block. One problem with this procedure is that the first block is less unique as subsequent blocks. If one encrypted message started the same as another message, the beginning blocks of the two messages would be the same. This would help an attacker to find out what the message(s) are. To solve this problem, you’ll create an initialization vector or an IV.

An IV is a block of random bytes that are XOR’d with the leading block of the data. All subsequent blocks are dependent on the previous block, so using an IV uniquely encrypts the entire message.

Inside encrypt(), below the last code block, add the following code to create an IV:

// 9
val ivRandom = SecureRandom()
// 10
val iv = ByteArray(16)
// 11
ivRandom.nextBytes(iv)
// 12
val ivSpec = IvParameterSpec(iv)

Here’s what’s happening:

9. Create a new SecureRandom object so that you’re not using a cached, seeded instance.
10. Create a byte array with a size of 16 to store the IV.
11. Populate iv with random bytes.
12. Create the IvParameterSpec with the random iv so that you can use it when performing the encryption.

Now, it’s time to use keySpec and ivSpec to encrypt a note. Below previously added block, add:

// 13
val cipher = Cipher.getInstance("AES/CBC/PKCS7Padding")
cipher.init(Cipher.ENCRYPT_MODE, keySpec, ivSpec)
// 14
val encrypted = cipher.doFinal(plainTextBytes)
// 15
map["salt"] = salt
map["iv"] = iv
map["encrypted"] = encrypted

Here’s the explanation:

13. Create and initialize the Cipher using AES/CBC/PKCS7Padding. This specifies AES encryption with cypher block chaining. PKCS7 refers to an established standard for padding data that doesn’t fit into the specified block size.
14. Encrypt the bytes of the data with the cipher.
15. Place the salt, iv, and encrypted bytes into a HashMap.

Now encrypt() can be used to encrypt the note text before writing to a file. To finish off the file encryption, insert the following code into addNote():

if (isExternalStorageWritable()) {
  ObjectOutputStream(noteFileOutputStream(note.fileName)).use { output ->
    output.writeObject(encrypt(note.noteText.toByteArray()))
  }
}

The code above creates an ObjectOutputStream with use() and utilizes it to write the encrypted message out to the file.

Lastly, you have to change the instance of the NoteRepository in MainActivity.kt to this:

private val repo: NoteRepository by lazy { EncryptedFileRepository(this) }

Build and run. Name a file as EncryptedTest.txt, add a random note and press WRITE to create an encrypted file. Open the file with Device File Explorer a notice that it’s filled with unreadable data because the file is encrypted.

Now you must add a mechanism to decrypt the file.

Decrypting the file

Locate decrypt() and add the following code in it:

var decrypted: ByteArray? = null
  try {
    // 1
    val salt = map["salt"]
    val iv = map["iv"]
    val encrypted = map["encrypted"]
    // 2 
    val passwordChar = passwordString.toCharArray()
    val pbKeySpec = PBEKeySpec(passwordChar, salt, 1324, 256)
    val secretKeyFactory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1")
    val keyBytes = secretKeyFactory.generateSecret(pbKeySpec).encoded
    val keySpec = SecretKeySpec(keyBytes, "AES")
    // 3 
    val cipher = Cipher.getInstance("AES/CBC/PKCS7Padding")
    val ivSpec = IvParameterSpec(iv)
    cipher.init(Cipher.DECRYPT_MODE, keySpec, ivSpec)
    decrypted = cipher.doFinal(encrypted)
  } catch (e: Exception) {
    Log.e("SIMPLENOTE", "decryption exception", e)
  }
  // 4
  return decrypted

Here’s what’s happening:

1. Retrieve the salt, iv and encrypted fields from the HashMap.
2. Regenerate the key from the password.
3. Decrypt the encrypted data.
4. Return the decrypted data.

Call decrypt() after reading the file in getNote().

val note = Note(fileName, "")
if (isExternalStorageReadable()) {
  // 1
  ObjectInputStream(noteFileInputStream(note.fileName)).use { stream ->
    // 2
    val mapFromFile = stream.readObject() as HashMap<String, ByteArray>
    // 3
    val decrypted = decrypt(mapFromFile)
    if (decrypted != null) {
      note.noteText = String(decrypted)
    }
  }
}
return note

Here’s how it works:

1. Open an ObjectInputStream with use() for reading data.
2. Read the data and store it into the HashMap.
3. Decrypt the file with decrypt(); if it was successful, assign the decrypted text to note.noteText.

Build and run. Notice the weird, encrypted data is now decrypted and understandable again! :]

Understanding Parcelization and Serialization

In computer science, marshaling is the process of transforming an object into a format that is suitable for transmission or storage. Android apps often transfer data from one activity to another, where Parcelization and Serialization are the means of marshaling objects.

By default — and because your app utilized ObjectOutputStream to write the file — the encrypted data, iv and salt values were all serialized when written to the file.

Serializable

Serializable is a standard Java tagging interface. This interface has no operation but it can be used to define the corresponding type as serializable. An object is serializable when it can be converted into an array of bytes and vice versa. If an object is Serializable it can also be written into a file or read from a file. However, when you restore an object you still need a compatible version of the class used during serialization. Implementing the Serializable interface isn’t enough - some properties are implicitly not serializable like Thread or InputStream. The Serialization process implies the use of reflection.

Reflection is the ability for an object to know things about itself. Therefore, using serializable can result in a lot of garbage collection which then translates to poor performance and battery drain. Fortunately, there’s another way to marshal objects in Android.

Parcelable

Parcelable is also an interface. However, unlike Serializable, Parcelable is part of the Android SDK. Because it’s designed not to use reflection, it requires the developer to be explicit about the marshaling process, making it more tedious to use. Despite this complication, using Parcelable can result in better app performance — although the gain in performance is usually imperceptible to the user.

Parcelable is often used when passing data between activities in a Bundle.

Note: The Parcelable API isn’t for general purposes. It’s designed to be a high-performance IPC transport. It’s not appropriate to place Parcel data into persistent storage.

Key points

• The internal storage is a great place to store files that are specific and private to your app.
• Use the internal cache to store temporary files.
• Use external storage to store files that you want users or other apps to have access to.
• External files are persistent, even after the app is uninstalled. However, internal files are deleted after uninstalling the app.
• To write files to the external storage, the correct permissions must be set in the manifest.
• Because the external storage isn’t secure, it’s a good idea to use AES encryption with a password-based generated key.
• Serializable and Parcelable are ways of marshaling data for transport or storage.

Where to go from here?

File encryption — and encryption in general — is a broad topic. To learn more about it, read the tutorial located at https://www.raywenderlich.com/778533-encryption-tutorial-for-android-getting-started. There is also a course on Jetpack Security https://www.raywenderlich.com/10135609-jetpack-security, which covers using Jetpack Security for the keychain and file encryption. To dig deeper into file management on Android, also read the official documentation available at https://developer.android.com/guide/topics/data/data-storage.