Category archives: Snaking

Discussions of programming, particularly of programming Python RSS

Save/Restore for Django "Embedded" Apps

So we have a lot of Django-based code where we'd like the user to be able to download a (subset of) the Django database as a "config file", then upload that "config file" to some number of other machines (or the original one), potentially long after the database has been migrated. I've got the skeleton of that working, I record all of the current migrations in the data-file, I play the migrations forward to that point, then attempt to insert the records, then continue migrating. That almost works, but inserting the stored records fails, as the standard "loaddata ...

Continue reading

Back to React Learning

I wanted to get back to doing a bit of React.js, so I started converting more of the WebToys into React (from Angular and raw Jquery). It took a bit of staring at old code to remember the React way of things. There are definitely some things I need to get sorted; how to parameterize from HTML to React cleanly, for instance (so an element can specify parameters for a view, for instance); it works, but not happy with the approach so far. The form story also seems a bit broken in my usage; it seems like 20-30 lines ...

Continue reading

Eventually Text Munging Gets Nasty

So today I started into the basic command-and-control part of the Listener system. The original tack I took was to create escaped commands inside the text stream using the "typing" substitutions (the stuff that converts ,comma into ','):

some text \action_do_something more text

But that got rather grotty rather fast when looking at corner cases (e.g. when you want to type \action to document that mechanism). So I reworked to have two different levels of operation, the first pre-processes to find commands and splits out the text such that you get a sequence of commands-and-text while interpreting. That should allow ...

Continue reading

First-Light on Dictating Code into Eric

So as of earlier this evening I've now got the Listener service hooked up such that I can dictate code into Eric (via an Eric Plug-in talking to the service over DBus). There's still an enormous amount that doesn't work. But that first-light moment has made me rather happy; instead of a collection of little tools/toys, there's something that has the rough shape of a working project.

The actual Eric code so far is about 150 lines, with a lot of that being boilerplate for an Eric plugin. I'll likely do quite a bit ...

Continue reading

Aw, come-on QtDBus, just work already

[Update] I got it working by going down to the "connection" level and registering the callback there. The code below is updated with the working version... as of now I've got some basic "voice coding" possible, but lots more to do to make it practical and useful.

Plowing ahead with integrating Listener into Eric (my primary IDE). That seemed to be going swimmingly, got a new plugin created, set it up to notice new editors, disconnect old ones and (in theory) process new results by inserting the recognized and interpreted text at the current position. Then I tried to ...

Continue reading

Interpretation of Dictation

Sometimes as you're developing a project it's easy to lose site of the end goal. I've spent so long on giving Listener context setup, code-to-speech converters, etc. that the actual point of the whole thing (i.e. letting you dictate text into an editor) hasn't actually been built. Today I started on that a bit more. I got the spike-test for a dbus service integrated into the main GUI application, so that you can actually get events from the same Listener that's showing you the app-tray icon and the results. I disabled the sending of ...

Continue reading

Using GStreamer AppSink from Python

So I decided to play a bit with how to get raw audio samples from gstreamer pipelines. This is normally done (in C or whatever) with an `appsink` element in your gstreamer pipeline. You hook up the pipeline, watch for the buffer, and then you can map it into a Numpy buffer (or whatever you like really). I haven't actually got a use-case for this right now; when I sat down I was toying with the idea of running neural nets on the samples to try to detect phonemes, but that will be some other day.

For today, consider ...

Continue reading

Python-dbus needs some non-trivial examples

So I got tired of paying work this afternoon and decided I would work on getting a dbus service started for Listener. The idea here is that there will be a DBus service which does all the context management, microphone setup, playback, etc and client software (such as the main GUI and apps that want to allow voice coding without going through low-level-grotty simulated typing) can use to interact with it.

But how does one go about exposing objects on DBus in the DBus-ian way? It *seems* that object-paths should produce a REST-like hierarchy where each object I want to ...

Continue reading

Seem to need hidden-markov-models for text extraction...

So in order to "seed" listener with text-as-it-would-be-spoken for coding, I've built up a tokenizer that will parse through a file and attempt to produce a model of what would have been said to dictate that text. The idea being that we want to generate a few hundred megabytes of sample statements that can then be used to generate a "python coding" or "javascript coding" language model. Thing is, this is actually a pretty grotty/nasty problem, particularly dealing with "run together" words, such as `asstring` or `mkdtemp`. You can either be very strict, and only allow specifically defined ...

Continue reading

Project-to-language model almost working

So at this point Listener can generate a language model from a (Python) project and the dictation results from that are reasonable, but by no means perfect, still, you can dictate text as long as you stick to the words and statements already used in the project.  There are some obvious reworks now showing up:

  • we need the dictionaries to chain, and we likely need to extract a base "Python" statement-set by processing a few hundred projects (yay open source)
  • we need to have separate statement, dictionary, etc. storage for the automatically generated and actual user-generated stuff so that on ...

Continue reading