ChatGPT Can Now See Your Screen on macOS But There is a Catch

Nov. 15, 2024

OpenAI has updated itsChatGPT desktop app for macOSand brought a new feature called “Work With Apps” in beta. It can read the content on your screen and helps you generate accurate answers by understanding the context. Keep in mind that it’s not a full-fledged screen-sharing feature with ChatGPT as promised earlier, but limited to coding apps right now.

The ChatGPT app on macOS can now see the content on compatible applications such as Xcode, TextEdit, iTerm, Terminal, VS Code, and other coding apps. To use it, open the compatible app and launch the ChatGPT app. You will see a “Work With Apps” button in the chat bar. Click on it and select the coding application.

You can now start prompting ChatGPT and ask contextual questions. You can also select lines in your code editor and ChatGPT will focus on those lines. To deliver this feature, ChatGPT uses macOS Accessibility API. You can enable or disable permission anytime you want.

Bear in mind that your interactions with ChatGPT while working with coding apps may be used to train AI models. If you are not comfortable with it, you candisable mode training on ChatGPT. So are you excited about the real-time ChatGPT coding companion or do you still preferCursoror other AI coding editors? Let us know in the comments below.

Passionate about Windows, ChromeOS, Android, security and privacy issues. Have a penchant to solve everyday computing problems.