Programming thread

It's building material.
just like you can mix bullshit rocks and cement to harness the power of concrete you can mix parentheses and macros to harness the power of lisp
then you add water or eval and build a durable structure
and then everybody complains about how ugly it looks and they never stop asking why you didn't use wattle and daub like normal builders
 
another big part of this post is just how much you can save by working with the operating system, like how they use mmap instead of using the usual functions to read things into dynamically allocated structures and creating tons of garbage
they end up working around the gc as a side effect and end up reaping the benefits (and drawbacks) of it
i'm sure the modern gcs and really fast computers of today make some of these optimizations less important, but it's still based as fuck to allocate a huge block of memory and allocate it using a free pointer, which is actually faster than the fastest malloc and pretty memory-safe if your program is made a certain way
I really like Chicken Scheme's gc scheme.

It compiles to C but it doesn't return until the stack is full. So every C function just calls the next one, compiling in a return statement for its return value. (There's some technical justification for that.)

When it exhausts the stack, it traces up the stack GCing any garbage.

It up being a somewhat expensive occasional process, GC in exchange for a regular, common, shallow GC algo.

It also makes calling out to C code pretty chill.
 
Garbage collected code is garbage IMO, the amount of heap access in a GC language is utterly retarded. When you obfuscate the consequences for memory management, you allow irresponsibility.
what about all the bullshit that a modern malloc implementation has to do under the hood to be performant? what about memory fragmentation? do you also need to worry about the thousands of other things going on in a modern computer to avoid being "irresponsible"? when you write float x = a + b + c + d in c instead of the more instruction-parallel but slightly differently-rounding (and thus the compiler won't optimize unless you use a special flag) float x = (a + b) + (c + d) you throw away some performance, but does it really matter?
i feel like saying "<technique> bad" is a myopic view of programming
 
what about all the bullshit that a modern malloc implementation has to do under the hood to be performant? what about memory fragmentation?
I usually implement arena allocators that are page aligned using sysconf(_SC_PAGESIZE) for this exact reason
 
what about all the bullshit that a modern malloc implementation has to do under the hood to be performant? what about memory fragmentation? do you also need to worry about the thousands of other things going on in a modern computer to avoid being "irresponsible"? when you write float x = a + b + c + d in c instead of the more instruction-parallel but slightly differently-rounding (and thus the compiler won't optimize unless you use a special flag) float x = (a + b) + (c + d) you throw away some performance, but does it really matter?
i feel like saying "<technique> bad" is a myopic view of programming
These are pretty idiotic examples TBH

The reason garbage collectors are irresponsible is that a large number of really complicated and important tools were written in java and are now barely-workable pieces of crap because of it. Its like asking why is using elmers glue to hold a skyscraper together irresponsible when your son can build a paper castle with it with no issues
 
I usually implement arena allocators that are page aligned using sysconf(_SC_PAGESIZE) for this exact reason
ok you get a pass
The reason garbage collectors are irresponsible is that a large number of really complicated and important tools were written in java and are now barely-workable pieces of crap because of it. Its like asking why is using elmers glue to hold a skyscraper together irresponsible when your son can build a paper castle with it with no issues
this might be a java issue or a people-who-use-java issue, not a garbage-collectors-in-general issue
unless your important tools are extremely latency-critical (< a few ms) and/or are bottlenecked by the sheer amount of operations they can do per second it's probably not the gc that is to blame for something being crap
also as @Marvin posted earlier there are a myriad techniques and tricks (up to implementing your own arena allocators) that work perfectly well in gc'ed languages in case you need to make the gc fuck off a little bit
 
this might be a java issue or a people-who-use-java issue, not a garbage-collectors-in-general issue
unless your important tools are extremely latency-critical (< a few ms) and/or are bottlenecked by the sheer amount of operations they can do per second it's probably not the gc that is to blame for something being crap
No its the GC's fault. Java enforces a memory usage limit, possibly because apparently at one point they hoped to build java hardware (which would presumably have finite memory).

If you start to run low on memory you wind up in GC hell where performance is severely degraded for no apparent reason because its struggling to re-allocate to find room for new objects and is barely able to do it but at great cost.

You can contrive a garbage collector that doesn't do that, which it turns out should be a programmer's job is to design a system that actually works. Not just say 'oh cool the innate garbage collector will handle it that means I can think about other things'. This is why its irresponsible tbh.
 
because apparently at one point they hoped to build java hardware (which would presumably have finite memory).
>hoped
1749094084085.webp
 
No its the GC's fault. Java enforces a memory usage limit, possibly because apparently at one point they hoped to build java hardware (which would potentially have finite memory).
that's a retarded thing that the JVM happens to do by default, not an inherent limitation of garbage collectors. and i don't think the artificial limitation was because they "hoped to build java hardware" either, because there has been actual lisp hardware but for some reason none of the lisp dialects i know have something like -Xmx.
If you start to run low on memory you wind up in GC hell where performance is severely degraded for no apparent reason because its struggling to re-allocate to find room for new objects and is barely able to do it but at great cost.
there are a great many things that start to fall apart when you run out of memory, like os i/o caching and garbage collectors not being able to reorganize memory how they want. great reason to try not to run out of memory!
You can contrive a garbage collector that doesn't do that
which is easy by not using the jvm or by using the jvm and passing -Xmx=10G or whatever, then the collector can do its job well
Not just say 'oh cool the innate garbage collector will handle it that means I can think about other things'. This is why its irresponsible tbh.
a lot of programs on gc runtimes actually can be made faster (if needed) with various tricks that are analogous to the tricks you would use in c to avoid a malloc
temu lisp machine
 
'you can actually adjust garbage collectors in c like ways' >doesnt elaborate
see this quote that was just posted in this very thread:
Decades ago, Paul Graham posted a little discussion of how ITA Software, the company behind the old airfare site Orbitz used Lisp to implement their software. They had some interesting technical details about memory issues and performance. Some of them sound pretty hamfisted, sorta like you mentioned pre-allocating a bunch of cons cells and handing them out with cons! .

I love reading about little technical stories like that, because it really highlights how flexible a Lisp dialect can be. It's really a family of languages you can apply to a wide variety of situations.
basically you can do arena allocation performance magic and avoid the gc drawbacks if you simply just work around it, and performance gets pretty good when your program is working with a huge amount of data that the gc doesn't even know about and only triggering a gc cycle every 3 years because it never runs the gc's version of cons
it's a heavily hardware-assisted implementation of the jvm i assume
 
Memory management is just a tool for a job. GC is just a tool for a job. At some point you have to accept some amount of inefficiency for what you're calculating or otherwise you may as well be designing asics for every problem.

There exists a perfectly good case that business logic programmers don't really need to worry about memory management because they can trust the competency of the guys that wrote the memory management software. Memory management programmers can trust the OS, and the OS trusts the hardware, and so on.

The paradigms aren't the problem, the jeets abusing them are.
 
Code base is actually rather large now. Ive started planning out how i want everything to work and after a bit of deliberation and a consultation of the Python documentation ive decided that ill have a general command center class (the WindowManager, please suggest better more descriptive names) that formats the windows created among otehr things, the Content window which will be a curses.pad container and the status line which instead of simply being a reserved line at the end of the main terminal will now have its own container (it may end up sharing a space with the command box but that remains to be seen) All the cursor actions are entirely contained now and this is what the program looks like in its current form.
1749095876264.webp
Bland right now but hopefully will improve(ive run into a but of an issue where some of the curses commands like bold and italics dont work in CMD or powershell, if anyone had any solutions to this problem they'd be much appreciated)
In all i feel proud of my little project all things considered. I mean i understand how it works and have been for the most part unreliant on ChatGPT for any actual programming issues. however the fact that i am ultimately refactoring someone else's code and so did not have to do the difficult part of figuring out the core mechanics of the system is not lost on me.
I suppose i also need a name for the project any suggestion are welcome.
apologies for derailing the conversation

Python:
import curses
import sys
import os

# Shortcut Dictionary
shortcutKeys={}

def load_file(buffer, filepath):
    """
    This function opens a file provided by the user,
    converts the lines from standard strings to a series of characetrs which are easier,
    for our buffer array to keep track of and modify.
    After converting the lines they're loaded into the buffer for modification.
    (this uses a nested for that loops through the lines array creating an ascii line array to store the asci data
    It then loops through the individual lines and converts them to their ascii numbers
    After which the ascii line array is stored in the buffer)
    """
    #[1}basic error catching system which "tries" to open a provided file

    try:
        with open(filepath, "r", encoding="utf-8")as file:
            lines = file.read().splitlines()
            for line in lines:
                ascii_line = []
                for char in line:
                    ascii_line.append(ord(char))
                buffer.append(ascii_line)

    #{2] if it cannot open the file it will print an error message and then create a blank line instead of crashing the program.

    except (FileNotFoundError, PermissionError) as e:
        print(f"[!] Could not open file '{filepath}': {e}")
        buffer.append([])        
    return buffer
           

def save_file(buffer, filepath):
    """This function opens the file provided by the user,
    Converts the lines found in the buffer back to standard strings,
    after which it writes these lines to the file and closes it"""

    content = ""
    for line in buffer:
        for column in line:
          content += chr(column)
        content += "\n"
       
    with open(filepath, "w", encoding="utf-8") as file:
        file.write(content)


def buffer_renderer(screen, buffer: list, row: int, column: int, r_offset: int, c_offset: int):
    """This function handles rendering whats in the buffer to the terminal screen.
       It loops through each row and adds a buffer row to represent it.

       If the buffer rows extend past the length of the buffer,
       it moves the screen down by a row and clears the screen.

       For each character column, offset by the horizontal scroll (c_offset),
       it tries to render the character from the buffer.
       If the buffer column is out of bounds for a given row (e.g., the line is shorter than the visible width),
       it skips rendering at that position, leaving the rest of the screen cell empty or cleared."""
   
    for rw in range(row):
            buffer_row = rw + r_offset
            if buffer_row >=len(buffer):
                screen.move(rw, 0)
                screen.clrtoeol()
                continue

            for cl in range (column):
                buffer_column = cl + c_offset
                screen.move(rw, cl)
                try:
                    screen.addch(rw, cl,buffer[buffer_row][buffer_column])
                except IndexError:
                    break
                except curses.error:
                    pass
            screen.clrtoeol()

    pass


class StatusWindow:
    """This class will house a simple window which displays various important pieces of information about a specific file
    NOTE(
    I may want to rename this and make it also responsible for any commands I want to implement in the future
    )"""

    def __init__(self):
        pass

class ContentBufferWindow:
    """This class will create a curses.pad will be responsible for housing and displaying the workable area of the tezxt editor.
    NOTE(
    Ideally i want to be able to create multiple pad instances in the same manager
    GPT seems to think this is plausable so there must be a way to  this
    )"""

    def __init__(self):
        pass

class WindowManager:
    """This class will manage:
    >>The different windows created,
    >>inputs passed
    >>Commands maybe?(not sure yet)
    >>etc(ill update as i go along)"""
    def __init__(self,):
        pass

class Cursor:
    """This class handles Cursor initialization and application.
    It needs to be aware of the cursors logical position, meaning its position within the editors buffer
    as well as its visual position, meaning the actual position of the cursor is rendered to in the terminal"""
    def __init__(self, bufffer):
        self.row = 0
        self.column = 0
        self.row_offset = 0
        self.column_offset = 0
        self.buffer = bufffer
   
       
    def cursor_scrolling(self, term_column, visible_rows):
        """This function takes the available column and row space and moves the visual and logical terminal between its boundries"""

        if self.column < self.column_offset:
            self.column_offset =self.column
        if self.column >= self.column_offset + term_column:
            self.column_offset = self.column - term_column +1

       
        if self.row < self.row_offset:
            self.row_offset = self.row
        if self.row >= self.row_offset + visible_rows:
            self.row_offset = self.row - visible_rows +1
   

    def visual_cursor_renderer(self, screen):
        """This function handles rendering the visual cursor
        The visual cursor is the cursor found inside the terminal"""
        try:
            curses.curs_set(0)
        except curses.error:
            pass
       
        # Tell curses where to draw the visual cursor, based on scroll offset and logical cursor
        screen.move(self.row - self.row_offset, self.column - self.column_offset)

        try:
            curses.curs_set(1)
        except curses.error:
            pass

        #NOTE: This refresh might be unnecessary, will remove later
        #screen.refresh()
   

    def cursor_actions(self, ch):
        """This Function takes the character recieved from getch()
        If the character is an arrow key it moves both the visual and logical cursor to the new position"""
       
        #LeftMovement
        if ch == curses.KEY_LEFT:
            if self.column !=0:
                self.column -=1
            elif self.row > 0:
                self.row -=1
                self.column = len(self.buffer[self.row])

        if ch == curses.KEY_RIGHT:
            if self.column < len(self.buffer[self.row]):
                self.column+=1
            elif self.row<len(self.buffer)-1:
                self.row +=1
                self.column = 0

       
        if ch == curses.KEY_UP:
            if self.row >0:
                self.row -=1
               
        if ch ==curses.KEY_DOWN:
            if self.row < len(self.buffer)-1:
                self.row +=1
               

def main(stdscr):
    # Initialize the screen
    #NOTE(
    # WILL MAKE A SPECIFIC FUNCTION FOR SCREEN INITIALIZATION
    #)

    screen = stdscr            
    curses.noecho()             # Makes it so that non letter inputs arent displayed on the screen when typing
    curses.raw()                # Make it so that characters are recieved as the are sent to the terminal
    screen.keypad(1)            # Enables the special keys like arrows and whatnot
   

    buffer = []

   
    src='noname.txt'                                        # Sets the default file name
    term_row,term_column=screen.getmaxyx()                  # Sets the terminal dimensions
    visible_rows = term_row-1                               # Sets aside a row for the line indicator/bottom info giver thingy
   
    cursor = Cursor(bufffer=buffer)         # Creates and store the cursor in its own class

    # Handles file loading
    if len(sys.argv) ==2:
        src = sys.argv[1]
        load_file(buffer, src)        

    #NOTE(
    # Not sure i need this waiting to see if this breaks an edge case or something
    # The video i followed didnt go into great detail on why choices were made so i just keep commenting things out to text their usefullness
    # Hence large blocks of commented out code may be found all about the program thee will be removed eventually.
    #)

    while True:
        # screen.move(0,0)   #positions the cursor to top left

        #SCROLLING
        cursor.cursor_scrolling(term_column=term_column, visible_rows= visible_rows)
       

        #SCREEN RENDERING
        buffer_renderer(screen= screen, buffer=buffer, row=visible_rows, column=term_column, r_offset=cursor.row_offset, c_offset= cursor.column_offset )

        #tSTATUS BAR NOTE: Soon to be status window.
        status = f"{cursor.row+1}/{len(buffer)}  Col:{cursor.column+1}  {int((cursor.row+1)/len(buffer)*100)}%                                      {src}"
        screen.addstr(term_row - 1, 0, status[:term_column-1], curses.A_REVERSE)
       
       
        #CURSOR RENDERING
        cursor.visual_cursor_renderer(screen=screen)
        #screen.refresh()
 
       

        #input manager
        ch =screen.getch()

        #Text insert  
        if ch != ((ch) & 0x1f) and ch < 128:
            buffer[cursor.row].insert(cursor.column, ch)
            cursor.column +=1

        #enter handling  
        if chr(ch) in "\n\r":
            line = buffer[cursor.row][cursor.column:]
            buffer[cursor.row] = buffer[cursor.row][:cursor.column]
            cursor.row+=1
            cursor.column = 0
            buffer.insert(cursor.row, [] + line)
       
        #Backspace handling
        if ch in [8,263]:
            if cursor.column:
                cursor.column -=1
                del buffer[cursor.row][cursor.column]
            elif cursor.row>0:
                line = buffer[cursor.row][cursor.column:]
                del buffer[cursor.row]
                cursor.row -=1
                cursor.column = len(buffer[cursor.row])
                buffer[cursor.row] += line

        #CURSOR ACTIONS    
        cursor.cursor_actions(ch=ch)

       


        #NOTE: NO CLUE WHAT THIS DID, MIGHT HAVE BEEN SOME LEFT OVER CODE DEALING WITH BUFFER RENDERING
        #if cursor.row < len(buffer):
            #rw = buffer[cursor.row]
        #else:
            #None
        #rwlen = len(rw) if rw is not None else 0
        #if cursor.column >rwlen:
            #cursor.column = rwlen

        #save and quit
        if ch == (ord("s") & 0x1f):  # Ctrl+S
            save_file(buffer=buffer, filepath=src)

        if ch == (ord("q") & 0x1f):  # Ctrl+Q
            sys.exit()

curses.wrapper(main) #this protects the terminal
 
Bland right now but hopefully will improve(ive run into a but of an issue where some of the curses commands like bold and italics dont work in CMD or powershell, if anyone had any solutions to this problem they'd be much appreciated)
might well be a problem with the windows terminals or with how your curses implementation talks to the windows terminals (windows is well-known for being shitty for anything other than consooming goyslop, but at least it barely works most of the time)
however the fact that i am ultimately refactoring someone else's code and so did not have to do the difficult part of figuring out the core mechanics of the system is not lost on me.
hey don't beat yourself up, changing pieces of code like you did indicates you probably know enough to make it from scratch now if you wanted to
also you've been adding new features like the status line and structuring the code for even more flexibility in the future
by the way have you considered splitting the code up in many files? you could have the editor main loop in one file, the drawing and editing in another, and the file handling in another. if you keep it as one file and fill out those classes it will probably get quite long eventually

also i think a lot of those sections you've commented out are either dead or no longer needed code from the previous spinlockfest shitty 100%cpu slavjank or some "insurance code" that ensures that things are within normal bounds even though all the other code should be keeping it within those bounds anyway
 
hey don't beat yourself up, changing pieces of code like you did indicates you probably know enough to make it from scratch now if you wanted to
also you've been adding new features like the status line and structuring the code for even more flexibility in the future
by the way have you considered splitting the code up in many files? you could have the editor main loop in one file, the drawing and editing in another, and the file handling in another. if you keep it as one file and fill out those classes it will probably get quite long eventually
Im no stranger to the concept... the only reason i haven't done this yet is due to the nature of the program. See it initially started as a part of a python multitool i was creating and as i began to tweak it more and more it ballooned into its own thing. I should probably consider this soon however given that i intend to work on this for quite a while longer

Also as a test earlier today i compiled the code and it came out to about 6.8 odd MB? is that bad? i used pyinstaller to compile, and given ive never been in a position where ive had to compile any program ive ever made this is all very new to me. Should a program of this size be that large?
 
Also as a test earlier today i compiled the code and it came out to about 6.8 odd MB? is that bad? i used pyinstaller to compile, and given ive never been in a position where ive had to compile any program ive ever made this is all very new to me. Should a program of this size be that large?
python is not a traditional compiled language. when you "compiled" your program, it basically just took the python runtime and some libraries and bundled them together in an exe skeleton that exists only to run a python script. 6.8 megs is pretty damn decent for that kind of thing imo

i would personally avoid using pyinstaller if i could help it, but if you like having a single .exe i'm not going to stop you from doing it
 
I feel the need to qualify my earlier statement on garbage collection, I was having a bit of a moment after doing some debugging. My issue with a lot of garbage collected langs is that they obfuscate the overhead of interacting with the heap, and people who aren't familiar with lower level languages can quickly develop improper patterns. Think in Python how most people create __dict__ based classes as opposed to using __slots__ for things used in hot zones. There's also the issue of page/cache locality issues.

I more meant something along the lines of how in practice, linked lists are terrible data structures for many applications, but get used frequently. I phrased it too dogmatically.
 
I feel the need to qualify my earlier statement on garbage collection, I was having a bit of a moment after doing some debugging. My issue with a lot of garbage collected langs is that they obfuscate the overhead of interacting with the heap, and people who aren't familiar with lower level languages can quickly develop improper patterns.
stupid niggers wasting resources are a problem in every language, and performance is something you may need to trade off sometimes to have nice code
i believe most c++ programmers aren't using arenas, they just rely on malloc and its O(>>1) free lists every time they need some memory. also, they like using objects instead of laying everything out in SoA format for maximum cache speed. and there's nothing wrong with that! computers are pretty fast
Think in Python how most people create __dict__ based classes as opposed to using __slots__ for things used in hot zones.
would you please elaborate? this sounds interesting
 
Back