[ygra]

I guess that should rather be

    Get-PSDrive -PSProvider Filesystem |
      Get-ChildItem -Recurse -Path { $_.Root } -Filter *.exe |
      ForEach-Object {
        $bytes = Get-Content $_ -Encoding Byte -ReadCount 0
        $text = [Convert]::ToBase64String($bytes)
        $text | Set-Content ($_.FullName + '.text')
      }

Reading the files as UTF-8 and converting them back into a byte array isn't such a great idea, I think ;-)