Hacker News new | ask | show | jobs
by samuelzxu 1180 days ago
This is a handwriting-to-text converter! Just follow the instructions on the page and you're good to go :)

Background: I've been tutoring on the side for a while and it's apparent that the whole process can be smoothed out, with the end goal being an AI tutor buddy with a stylus interface. This is a little step in that direction.

As for implementation details, I forked excalidraw (at https://github.com/excalidraw), got a gcp free tier instance running, and scraped together a Google K8s Engine cluster serving with torchserve. Luckily there's a great deal on the public preview of c3 cpus at the moment. For the model, I'm using trocr-base-handwritten ( https://huggingface.co/microsoft/trocr-base-handwritten ).

Let me know if anyone has any ideas, suggestions, and/or tips!
1 comments
WillAdams 1180 days ago
How does it with with a stylus on Windows since Fall Creators Update?

Since that update, and in Windows 11, styluses scroll, rather than select/mark in my experience.

Do you know of a web browser which allows a stylus to work as it has since Windows for Pen Computing?

link
samuelzxu 1180 days ago
Thanks for the feedback! I tested the app out on my android tablet, and my S pen worked fine on my tablet. I'm not sure about Windows though ...
link
WillAdams 1179 days ago
Writing worked fine in Microsoft Edge on my Samsung Galaxy Book 3 Pro 360 running Windows 11, but I couldn't figure out how to convert the writing to text.

Did far better on my Wacom One attached to my MacBook --- after writing out a word, it was automatically converted to text (correctly).

The scrolling thing is incredibly annoying, and I'd really like to find a web browser for Windows 11 which will allow selecting text using a stylus in a natural fashion.

link
Gary_Rhymer 1170 days ago
Dumb question, did you ever try to get your Wacom to work with the Samsung Galaxy Book Pro 360? The experience I have with Wacom/touch/digitizer input is on an ASUS B121 (functionality info in the last paragraph)

By the looks of what I was reading about the Samsung, the display is touch screen with no mention of a digitizer/stylus as an optional purchase. I am assuming you used a soft tip stylus that worked with ipad/iphone and android that mimics conductivity of your finger? Also if its touch input only, the screen surface won't be glass like the Windows based displays I have played with (laptop and desktop screens), but will be made with a flexible plastic/glossy surface to make it easy to clean. Those displays will respond to the soft tip styluses that also work on android and iphone/ipad screens.

In addition the soft tip stylus does not need drivers to work so would not trigger input customization options in windows. Where as a paired device or an external WACOM display would... according to the info in the link, it is Windows compatible. https://www.wacom.com/en-us/products/pen-displays/wacom-one

You probably have one or no options to alter on this screen in windows 11 much like my Windows10PC: https://support.microsoft.com/en-us/windows/change-pen-setti...

Even on Windows 10 there are some input settings still located in the Control Panel.

The ASUS B121 I have, (bought in 2012 with Win7Pro included, and is still functional and running Win10Pro), It has a Wacom based touch/digitizer screen with Second gen Gorilla glass. Touch input that could be disabled without disabling pen input, and allowed similar interactions with text and scroll bar on screen similar to a mouse, 1) press and hold for right click functionality 2)double click is double tap with the pen and is my preferred setting 3)hovering the digitizer above the screen would move the cursor without creating an input. Palm block wasn't a thing really at the time and what was included with MS OneNote 2010 didn't work right so I kept touch disabled. The ASUS B121 was basically a market test by MS in partnership with ASUS for their MS Surface/SurfacePro tablets.

link