Research And Development

I write a bit about the some of the challenges I encountered with implementing the Chat App & IPC (Inter-process-communication. I also talk about Lua and how it can interop with the OS long term.

March 3rd - Dialogue Early Exit

A lot of the development lately has been implementing Lua, learning it, and sandboxing it correctly. So I’ve been doing a lot of Lua specific research.

One of the things a dialogue system needs is the ability to exit via both code, and from the user. Either the user will walk away, exit the dialogue prompt, or in my case, close the exit button.

March 3rd - Lua Modding License

Lua File Modding License is an idea I’ve been considering - I figured specific Lua files that are part of the core API need a license that allow modding. This way users can freely patch the Lua file itself, and share that.

In my mind a core component of modding is going to be copying existing code, sharing modifications on existing pieces of code that are meant to fix issues people have experienced.

My game isn’t going to be that complex, so a good chunk of it can be written in Lua. At least the dialogue portions.

In my mind having a clear cut license that allows users to modify the code, and upload their own patches to pre-existing work is important.

I do not know of any games that are doing this, but I doubt I am the first. It seems very obvious to me, to allow users explicit rights to the code. At least, parts of them. Certain files I will keep private, but the core API should be public. Allowing users to extend the API as they see fit.

Obviously there needs to be limits, I still retain the copyright to all code I write. Along with any dialogue or assets created for the game. Outside of that though, the raw Lua code itself I believe could be shared, at least in the context of modding this specific game.

March 7th - ChatApp Demo Mistakes

Oversight #1

I’ve been working on bridging the Lua code from the little chat app demo, into a multi chat rough draft. This initial POC worked until I realized we needed multiple chats running at once. This is not an issue, but I didn’t think of it while using the simple single chat demo I wrote to create the API and integrate Lua.

Oversight #2

Data is queued as a runnable task with no information, this worked for my simple demo to integrate the API, but now for the complex system I wrote above, we need to peek the data and have access to that information. Not just defer execution of functions with it. I converted the system over to a message queue and now I can peek the data as I need.

Now everything is working as expected, I can queue the chatapp functionality in the background while I execute the message queue of the various dialogue script as I load them in on demand, into the same Lua environment - It’s a global environment that I intend to re-use as you open the chatapp.

My idea is specific apps cannot be opened multiple times, this will be one of them.

Oversight #3

This time it was messaging API. Again, with single chats I had imagined I could simply juggle the state, and have a single “reply” function. This is obviously false, and I couldn’t think of a better way to hack in the identifier so instead I decided to change the API one last time:

  -- greeting

  chatapp.to_example "Hello World"

  chatapp.from_example "\"Hello %player.name%\""

For this the main change being a to_ and from_ function now for each message. This honestly makes the DSL slightly easier to read, as it’s more akin to chat logs.

The queue contains the data fully, I just need to replay it. First, we need to capture the keyboard events, if one is there attempt to use the queue and start the phase 1, then when sent, we move to phase 2 - replay.

If a chat contains no queue, type as normal and we can have some dummy chatbot response to spook anyone who forces an invalid chat to load.

March 9th - IPC CMD

Today started with “When the OS boots, can I have an internal CMD prompt that is constantly running that can be queried by IPC?” - The answer to that question is yes.

In my OS, IPC stands for inter-process-communication. It’s essentially the highway for processes communicating with one another. These can be global processes, or singleton style processes where only one can be alive at once. The idea is this acts like a programmatic command line to add and execute specific commands.

You can execute any IPC command from the command line by typing ipc and then the address, followed by the packet you want to send. Strings require no special formatting, but everything else does.

To make the Lua os.execute function work, I figured we needed a programmatic way to access the command line from the IPC system. I did this by having an invisible - (kind of, it has no GUI) - command prompt that sits in the background, waiting for commands to come in. When the command is ran, it will clear the console screen, then return the results.

This works well, but theres a clear pitfall in my execution stack, I never added return statuses for my processes. In my mind we don’t need it, but it would be nice to implement.

Instead of just returning an integer, I figured why not return the results of the IPC call instead, which would just be the results of the command line itself. This is basically the concept of program pipes.

Now we have a one way method of passing data from program to program, rather than using files to achieve this, we can use the program results directly.

Obviously this will need to be cleaned up a bit, but the proof of concept seems promising.

When the results get cleaned up, we would in theory have two way communication. One from the newly formed output, but one as well from the arguments of the program on start.

ipc CMD echo example

Response: (example)

As you can see in the photo, there’s still some work to do:

I it integrated with os.ipc as a function that returns a string based on the results of the IPC command you executed.

One really cool IPC I made early on, is the GPU controller. Essentially this allows us to turn on specific shaders and control the state of them. I created a tiny demo that allows us to turn the shaders on, and transition into a retro PC look.

os.ipc("GPU", "b:true", "l:1000", "crt.f")

The system needs a bit of work, but it’s essentially exactly functional as the command line variant is:

os.execute("ipc GPU b:true l:1000 crt.f")

As a bonus I wrote os.gpi and the various functions that would be relative - but it’s not really needed. I prefer using the command line when I can, as I think it would help encourage exploration for modders, instead of hiding it behind layers of abstraction.

March 11th - Lua OS

Early in development, at some point in December of 2024, I wrote a dev-note where I theorized the concept of putting in Lua as a replacement for BATCH. I’ve been heavily thinking of this for the past few days since I’ve added the Lua dialogue.

I like to imagine a computer, where all remote code execution was only possible with interpreted languages, such as Lua or Python, etc. - You would have an ecosystem built specific for that platform.

Rather than being a binary trust world (trusting, trust) - you would have direct view of all the source code. The ability to audit it would be directly at your fingertips. Assuming your interpreter was not compromised, this would be an extremely safe way of handling code execution.

Internally - we have a core library I compile against, then I would write my specific app using it. Then I export the compiled binary, and then move it into the OS directory.

This is currently how we manage whole modules for the game, I break the large parts up and create whole modules that tackle individual tasks. Lua, GPU, Theming etc.

My system works well for the platform my system executes on, but we can build an entirely new platform - existing on top of this framework - that doesn’t have the limitations our current ecosystem does. This would be a large undertaking and probably something that would need to be post release for the full scope, but a small initial BATCH replacement at first is exactly what we currently have.

The sandbox is still in the works, after I add Lua choices I’m going to pause work on the Lua BATCH replacement and leave it for another day. I have to focus on the next few major milestones I have internally in my head before I can dedicate time to that.

March 14th - IPC Is Not Async

I’ve been bug fixing random things - the os.execute IPC integration failed for functions that were used to fetch data. This is because I was executing the IPC function async. Removing that code and making it blocking again, and it worked as expected. I need to make it a callback API if I want to make it async, for now I won’t bother with it, as we’re running the entire Lua stack on the async thread anyways.

Debug is broken. So is Error logging. Internally in the game. I don’t know when this broke, but it sucks. The feature fully integrated with the game log and was able to read system.out and system.err without issue. I’ll fix it eventually.

March 18th - Lua Friend Requests

I solved dynamic friend requests in Lua by just defining a Boolean function in the friends table as a variable. True means you can add them. False means it’s going to remain pending until that condition is met.

I want to extend the messaging to be on the IPC system:

-- Opens the chat window if it's not open yet, and starts the chat

ipc CHATAPP send jerry "Hello Jerry"

-- Receive the message back, if the window was closed it will re-open and possible notify on desktop depending on if the app is open or not

ipc CHATAPP receive jerry "Hello {%user%}"

-- Opens the chat window if it's not open yet, and starts the chat

ipc CHATAPP choice jerry "Yes", "chat/_jerry/_choice/_lua/_yes", "No" "chat/_jerry/_choice/_lua/_no"

March 21st - Bootstrapped Development

I’ve set up a git server, it’s an entire VPS dedicated to hosting the game code. There is a risk self hosting - but I feel I have no choice as I want to have files much greater than 100mb using GitLFS and not have to be billed per gigabyte.

The next step after this would be getting it hooked up to some kind of git windows runner so we could have automatic window builds. Realistically I’ll be compiling the release build manually for quite some time, but that’s okay.

March 23rd - Operating System Manual

I’ve decided to start to maintain a user manual, with 3 sections targeting 3 separate audiences.

• The OS User Manual (The Tome) The manual meant for new users - to teach the bare bones basics. This is not meant to be related to the game, but rather the core OS itself. This manual will most likely be written at a later date.
• The Modders Manual (The Grimoire) This is meant to be read by anyone who is interested in creating custom content for my OS game.
• The Internal Manual (The Codex) This is meant to keep future me in the loop with internal engine changes I’ve made. This project is massive and it’s a lot for me to juggle at once. As I transition between the various “hats” I have to wear during development, I will forget concepts, theories, ideas and todos. This manual is meant to keep me in the loop for API design choices, so I know why I did it a specific way.

March 25th - Hyperlinks

The concept for the game that I have in my head involves downloading as part of the storyline. Eventually there should be a downloading app, an installer, etc.

This means figuring out how the prompt for downloading will work. I’ve thought of 3 ways I can accomplish this.

1. I could simply re-use the existing choice system. I think this is good on the surface, as it allows users to preform choices from within a uniform area, one single GUI. All my other options are based within the chat window itself, not the choice window.
2. Hyperlinks - this was my initial idea, kind of how I send downloads to my friends today. The downside to this is you would generally expect a web browser, and some kind of thing to show the download progress, even if it’s just quick. That’s kind of out of scope, but I really like this idea. I want to add it still, and even include one for media so you can send pictures and click them like hyperlinks, and the dialogue could continue based on that. Not sure what use it would have yet but I was thinking two doors, or two pills, some kind of matrix reference.

  -- URL concept

  url("Title", function()
    if(ipc.getInt("demo.tutorial.game.install.state") == 1) then
      return
    end

    sleep(2000)
    ipc.setInt("demo.tutorial.game.install.state", 1)
    os.execute("ipc launch game.a")
  end)

3. Downloading - this is the last concept I thought of, you have the file name, total bytes, then the milliseconds it takes to download. It would show a progress bar, then a small ‘Open’ button would pop up. Not sure how viable this is as we’re using fixed components currently, we’ll see how hackable it is.

  -- DOWNLOAD concept
  download("Title", 3938474, 5000, function()
    --TODO we would prompt an installer normally
    if(ipc.getInt("demo.tutorial.game.install.state") == 1) then
      return
    end

    sleep(2000)
    ipc.setInt("demo.tutorial.game.install.state", 1)
    os.execute("ipc launch game.a")
  end)

I think adding 2 & 3 would be good for prototyping, and since I don’t have a concrete answer experimenting would be good. In the end having all of these mechanisms would be good as it gives creative freedom (to an extent) when writing the story. Obviously we need a way for apps to be installed, I think having dialogue manage that is an important step…

March 26th - ChatApp Integration

Lua Images are fully working, hyperlinks work as intended. I decided to scrap the download thru chatapp concept for now, as I think having a downloader app would be more in line with the concept.

That, and I think it will be much easier to just make an app.

The downside is, I thought of this really cool Lua syntax that would allow me to give dynamic programing into hardcoded specific applications. So this means lua chatapp can have access to specific applications, or at least support them being launched this specific way:

 -- URL concept with downloader
  hyperlink("https://13.38.36.33/game+crack.exe", function() --onClick function
    os.app("downloader", 3938474, 10000, function() --open function
      if(ipc.getInt("demo.tutorial.game.install.state") == 1) then
	      os.execute("ipc app gamelauncher game.a")
        return
      end

      sleep(2000)
      os.execute("ipc app install gamelauncher")
      os.execute("ipc game install game.a")
      os.execute("ipc notification New App Installed: GameLauncher")
      os.execute("ipc notification New Game Installed:Ggame.A")
      ipc.setInt("demo.tutorial.game.install.state", 1)
      os.execute("ipc app gamelauncher game.a")
    end)
  end)

The idea behind this, is when you click the hyperlink, the download UI pops up. Then you can click ‘Open’ and it will install and auto launch. If installed already, it will auto launch.

I’m enjoying putting large bits of code into Lua even though it requires manual interaction, I think it will let me be flexible in the end with a story, and be able to re-use the APIs for other various Lua based interactions.

My goal is to finish the ChatApp today, all we have left technically is the downloading mechanism, then the hinting system, then maybe some kind of re-polling regularly so the user state can be updated outside of the Lua runtime.

In other news, app launching is now standardized. When you launch an app in the code, it will return a controller for the process and such now.

March 31st - Month End Summary

332 Completed Tasks

Not as many as I’d like, but they are some massive changes. This month has been more about the research part of R&D, rather than development. Finding out what will work, etc. - I have been experimenting with smaller projects to tackle bigger aspects of this game.

2D GPU OS Renderer Update

The GPU Render is now looking 1:1 accurate and there are no clicking issues. I had introduced a scroll issue where the Y isn’t correct for some reason but that’s the only real major issue with the system.

GPU shaders is still a work in progress for translating mouse X,Y. I’ve decided to move on and try getting the core gameplay started so we can have an actual demo.

Gameplay Plans

A lot of this month has been research. Research how I should do gameplay, research how I should do dialogue. My original concept was to create a dialogue interpreter with our own custom language, but I’m experienced enough to know that the logical handling of that can get tricky and we’re better off using an implementation that is well tested, so we can focus on results instead of language building. This same thought applied to gameplay - is the game I’m making going to be fun? My original rendition was going to include many choice dialogue options, cause the more complex the story, the better, right? Wrong. I believe the story could be even better with minimal choices, if any.

Scripting Plans

The most important part of any game is going to be the scripting, what drives the story forward. How do you programmatically define the way the game is going to change. In this game, everything is reactive essentially - we’re creating the GUI for the user to explore. And they will tell their own story from this experience. Having a pre-baked story is key for them to guide themselves, but ultimately it’s about going to be like stepping into an unknown room, finding a bookshelf, then looking through it. The idea is to create a gameplay loop that can be controlled either by player actions, or by player conversations. I decided to focus on the first one which is easy to control, dialogue. I’ve got a background in creating dialogue systems, so I felt this would be a really fun challenge to tackle.

Dialogue Lua Plans

I’ve started the Lua implementation and gotten dialogue on it’s way to being possible. I made a chat app and a demo app to test the API while we interop everything with Lua. Lots of work but we’re making real progress.