[BUG] Command python.exe -m rich.emoji throws an error when the output is redirected to a file

[asourabh-airnav]

• I've checked docs and closed issues for possible solutions.
• I can't find my issue in the FAQ.

On Windows PowerShell or CMD, python.exe -m rich.emoji | Out-File -encoding unicode ".\richEmoji" causes error

I am trying to create a file having list of emojis and the text to invoke them. For this i tried redirecting the output of recommended python.exe -m rich.emoji to a file with a pipeline. But this causes the error UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f44d' in position 0: character maps to <undefined>

Here is the output:

PS C:\Users\xxxx.xxxx\dsc-template\.pixi\envs\default> .\python.exe -m rich.emoji | Out-File ".\richEmoji"
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\xxxx.xxxx\dsc-template\.pixi\envs\default\Lib\site-packages\rich\emoji.py", line 94, in <module>
    console.print(columns)
  File "C:\Users\xxxx.xxxx\dsc-template\.pixi\envs\default\Lib\site-packages\rich\console.py", line 1692, in print
    with self:
  File "C:\Users\xxxx.xxxx\dsc-template\.pixi\envs\default\Lib\site-packages\rich\console.py", line 866, in __exit__
    self._exit_buffer()
  File "C:\Users\xxxx.xxxx\dsc-template\.pixi\envs\default\Lib\site-packages\rich\console.py", line 824, in _exit_buffer
    self._check_buffer()
  File "C:\Users\xxxx.xxxx\dsc-template\.pixi\envs\default\Lib\site-packages\rich\console.py", line 2033, in _check_buffer
    self._write_buffer()
  File "C:\Users\xxxx.xxxx\dsc-template\.pixi\envs\default\Lib\site-packages\rich\console.py", line 2069, in _write_buffer
    legacy_windows_render(buffer, LegacyWindowsTerm(self.file))
  File "C:\Users\xxxx.xxxx\dsc-template\.pixi\envs\default\Lib\site-packages\rich\_windows_renderer.py", line 19, in legacy_windows_render
    term.write_text(text)
  File "C:\Users\xxxx.xxxx\dsc-template\.pixi\envs\default\Lib\site-packages\rich\_win32_console.py", line 402, in write_text
    self.write(text)
  File "C:\Users\xxxx.xxxx\dsc-template\.pixi\envs\default\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f44d' in position 0: character maps to <undefined>

Removing the pipeline results in the emojis being returned normally.

Platform

Click to expand

What platform (Win/Linux/Mac) are you running on? What terminal software are you using?
Windows 11 Enterprise with PoweShell terminal

I may ask you to copy and paste the output of the following commands. It may save some time if you do it now.

If you're using Rich in a terminal:

python -m rich.diagnose

╭───────── <class 'rich.console.Console'> ─────────╮
│ A high level console interface.                  │
│                                                  │
│ ╭──────────────────────────────────────────────╮ │
│ │ <console width=52 ColorSystem.TRUECOLOR>     │ │
│ ╰──────────────────────────────────────────────╯ │
│                                                  │
│     color_system = 'truecolor'                   │
│         encoding = 'utf-8'                       │
│             file = <_io.TextIOWrapper            │
│                    name='<stdout>' mode='w'      │
│                    encoding='utf-8'>             │
│           height = 34                            │
│    is_alt_screen = False                         │
│ is_dumb_terminal = False                         │
│   is_interactive = True                          │
│       is_jupyter = False                         │
│      is_terminal = True                          │
│   legacy_windows = False                         │
│         no_color = False                         │
│          options = ConsoleOptions(               │
│                        size=ConsoleDimensions(   │
│                            width=52,             │
│                            height=34             │
│                        ),                        │
│                        legacy_windows=False,     │
│                        min_width=1,              │
│                        max_width=52,             │
│                        is_terminal=True,         │
│                        encoding='utf-8',         │
│                        max_height=34,            │
│                        justify=None,             │
│                        overflow=None,            │
│                        no_wrap=False,            │
│                        highlight=None,           │
│                        markup=None,              │
│                        height=None               │
│                    )                             │
│            quiet = False                         │
│           record = False                         │
│         safe_box = True                          │
│             size = ConsoleDimensions(            │
│                        width=52,                 │
│                        height=34                 │
│                    )                             │
│        soft_wrap = False                         │
│           stderr = False                         │
│            style = None                          │
│         tab_size = 8                             │
│            width = 52                            │
╰──────────────────────────────────────────────────╯
╭─ <class 'rich._windows.WindowsConsoleFeatures'> ─╮
│ Windows features available.                      │
│                                                  │
│ ╭──────────────────────────────────────────────╮ │
│ │ WindowsConsoleFeatures(                      │ │
│ │ │   vt=True,                                 │ │
│ │ │   truecolor=True                           │ │
│ │ )                                            │ │
│ ╰──────────────────────────────────────────────╯ │
│                                                  │
│ truecolor = True                                 │
│        vt = True                                 │
╰──────────────────────────────────────────────────╯
╭────── Environment Variables ───────╮
│ {                                  │
│     'CLICOLOR': None,              │
│     'COLORTERM': None,             │
│     'COLUMNS': None,               │
│     'JPY_PARENT_PID': None,        │
│     'JUPYTER_COLUMNS': None,       │
│     'JUPYTER_LINES': None,         │
│     'LINES': None,                 │
│     'NO_COLOR': None,              │
│     'TERM_PROGRAM': None,          │
│     'TERM': None,                  │
│     'TTY_COMPATIBLE': None,        │
│     'VSCODE_VERBOSE_LOGGING': None │
│ }                                  │
╰────────────────────────────────────╯
platform="Windows"
```

</details>

[github-actions[bot]]

We found the following entries in the FAQ which you may find helpful:

• python -m rich.spinner shows extra lines.
• Strange colors in console output.
• How do I log a renderable?

Feel free to close this issue if you found an answer in the FAQ. Otherwise, please give us a little time to review.

This is an automated reply, generated by FAQtory

[willmcgugan]

It appears as though the encoding defaults to cp1252 when piping, which can't handle the emojis. You could try setting PYTHONIOENCODING env var to "utf-8".

[asourabh-airnav]

Does encoding change when output is piped? Without piping, the command works and i can see the emojis on terminal. Why should redirecting the output from console to a file cause python encoding to change?