Programming thread

  • Want to keep track of this thread?
    Accounts can bookmark posts, watch threads for updates, and jump back to where you stopped reading.
    Create account
Marvin said:
Well also I should probably also look at what Go or Python are doing for their CapnProto generators. Get an idea of what sort of features a typical generated API supports.
Click to expand...
For Go, it's almost certainly something with reflection. It's the closest built-in thing to anything resembling meta-programming in Go imo.

Marvin said:
Probably the most complicated syntax is the quasiquote syntax. Quasiquote lets you create a complicated list tree with individual elements substituted in with evaluated code. So `(a (list here) and 3 plus 3 is ,(+ 3 3)) evaluates to (a (list here) and 3 plus 3 is 6). The comma is "unquote" and it selectively evaluates the expression. The comma and the @ symbol in the enum code above is unquote-splice, which expects that the expression will return a list, and it expands the list inside the parent quasiquote.

A little confusing, but it makes the enum code (and any kind of macro code) a lot easier to write. As always, balancing terseness and readability.
Click to expand...
I love quasiquotes. Aside from shorthand, there is so much immense power when used with pattern matching.
y a t s said:
here's a quick and shitty JSON parser I wrote completely from scratch in Racket a few months ago for fun:
Code: 
#lang racket

;; (char-numeric?) includes other unicode numerals.
;; We only use American Numerals™ here.
(define (char-digit? c)
  (and (char? c)
       (char>=? c #\0)
       (char<=? c #\9)))

;; A simple wrapper for (char-whitespace?) with a type check.
(define (char-ws? c)
  (and (char? c)
       (char-whitespace? c)))

(define (char->token c)
  (match c
    [#\[ 'LBRACK]
    [#\] 'RBRACK]
    [#\{ 'LCURLY]
    [#\} 'RCURLY]
    [#\: 'KSEP]
    [#\, 'VSEP]
    [#\" 'QUOTE]
    [#\' 'APOS]
    [x x]))

;; Simple parser continuation.
;; lst: remaining tokens.
;; res: result upon return.
(struct cont (lst res))

(define (unwrap cn)
  (match cn
    [(cont _ r) (car r)]))

(define (bad-brack brack ex)
  (eprintf "Unexpected bracket: ~v. Expected: ~v.\n" brack ex))

(define (begin-arr tokens)
  (define (end-arr lst res)
    (cont lst `((arr ,(reverse res)))))

  (let loop ([tokens tokens]
             [bs '()]
             [res '()])
    (match tokens
      ['() (cont '() `(,(reverse res)))]
      [`(RBRACK ,_ ...) (end-arr (cdr tokens) res)]
      [`(VSEP ,_ ...) (loop (cdr tokens) bs (cons (void) res))]
      [_ (let ([lres (parse-json/r tokens)])
           (match (cont-lst lres)
             [`(VSEP ,r ...) (loop r bs (cons (unwrap lres) res))]
             [`(RBRACK ,_ ...) (loop (cont-lst lres) bs (cons (unwrap lres) res))]
             [x (eprintf "Expected separator or closing bracket after value. Found ~v\n" x)]))])))

(define (char->digit c)
  (- (char->integer c) 48))

(define (begin-numeric tokens)
  (define (get-base n)
    (if (zero? n) 1
        (add1 (floor (log n 10)))))

  (define (end-numeric lst res)
    (cont lst `(,res)))

  (let loop ([tokens tokens]
             [dec #f]
             [res 0])
    (match tokens
      ['() (cont '() '(res))]
      [`(,(? char-digit?) ,_ ...) (loop (cdr tokens)
                                        dec
                                        (+ (* res 10)
                                           (char->digit (car tokens))))]
      ;; Decimals.
      ;; Process RHS as its own int value and then add it to res after scaling.
      ;; dec acts as a flag, helping to avoid processing additional decimal points.
      [`(#\. ,_ ...) (if (not dec) (let* ([rhs (loop (cdr tokens) #t 0)]
                                          [ex (get-base (unwrap rhs))])
                                     (end-numeric (cont-lst rhs)
                                                  (+ res (/ (unwrap rhs)
                                                            (expt 10. ex)))))
                         (error "Unexpected additional decimal."))]
      [_ (end-numeric tokens res)])))

(define (begin-str tokens)
  (define (quote-match? q qs)
    (and (not (null? qs))
         (symbol=? q (car qs))))

  (define (end-str lst res)
    (cont lst `(,(list->string (reverse res)))))

  (let loop ([tokens tokens]
             [qs '()]
             [res '()])
    (match tokens
      ['() (error "Reached EOF while parsing string value.")]
      [`(,(or 'APOS 'QUOTE) ,_ ...) (cond
                                      [(null? qs) (loop (cdr tokens) (cons (car tokens) qs) res)]
                                      [(quote-match? (car tokens) qs) (end-str (cdr tokens) res)]
                                      [else (loop (cdr tokens) qs (cons (car tokens) res))])]
      [`(#\\ ,(or #\' #\" #\\) ,r ...) (loop r qs (cons (cadr tokens) res))]
      [_ (loop (cdr tokens) qs (cons (car tokens) res))])))

;; key-value pair.
(struct kvp (k v))

(define (begin-obj tokens)
  (define (end-obj lst res)
    (cont lst `((obj ,(reverse res)))))

  (define (parse-kvp tokens)
    (match tokens
      [`(,(or 'APOS 'QUOTE) ,_ ...) (let* ([kres (parse-json/r tokens)] ; parse key
                                           [vres (parse-json/r (cdr (cont-lst kres)))]) ; parse value
                                      (cont (cont-lst vres)
                                            `(,(kvp (unwrap kres)
                                                    (unwrap vres)))))]
      [_ (eprintf ("Expected key-value pair. Found ~v\n" tokens))]))

  (let loop ([tokens tokens]
             [bs '()]
             [res '()])
    (match tokens
      ['() (end-obj tokens res)]
      [`(RCURLY ,_ ...) (end-obj (cdr tokens) res)]
      [`(RBRACK ,_ ...) (bad-brack (car tokens) 'NONE)]
      [_ (let ([lres (parse-kvp tokens)])
           (match (cont-lst lres)
             [`(VSEP ,r ...) (loop r bs (cons (unwrap lres) res))]
             [`(RCURLY ,_ ...) (loop (cont-lst lres) bs (cons (unwrap lres) res))]
             [x (eprintf "Expected separator or closing brace after kvp. Found ~v\n" x)]))])))

(define (parse-json/r tokens)
  ;; Check for special keyword values like Infinity, NaN, and bools.
  (define (parse-special tokens)
    (match tokens
      ;; Infinity
      [`(#\I #\n #\f #\i #\n #\i #\t #\y ,r ...) (cont r '(+inf.f))]
      ;; -Infinity
      [`(#\- #\I #\n #\f #\i #\n #\i #\t #\y ,r ...) (cont r '(-inf.f))]
      ;; NaN
      [`(#\N #\a #\N ,r ...) (cont r '(+nan.f))]
      ;; true
      [`(#\t #\r #\u #\e ,r ...) (cont r '(#t))]
      ;; false
      [`(#\f #\a #\l #\s #\e ,r ...) (cont r '(#f))]
      [x (eprintf "Unexpected token: ~v\n" x)]))

  (match tokens
    ['() (cont '() '())]
    ;; Arrays.
    [`(LBRACK ,_ ...) (begin-arr (cdr tokens))]
    ;; Objects.
    [`(LCURLY ,_ ...) (begin-obj (cdr tokens))]
    ;; Strings.
    [`(,(or 'QUOTE 'APOS) ,_ ...) (begin-str tokens)]
    ;; Positive numbers.
    [`(,(? char-digit?) ,_ ...) (begin-numeric tokens)]
    ;; Negative numbers.
    [`(#\- ,(? char-digit?) ,_ ...) (let ([lres (begin-numeric (cdr tokens))]) ; cdr to drop negative sign.
                                      (cont (cont-lst lres) `(,(* (unwrap lres) -1))))]
    [_ (parse-special tokens)]))

(define (parse-json tokens)
  ;; Unwrap parse result.
  (unwrap (parse-json/r tokens)))

(define (tokenize str)
  (define (quote-match? q qs)
    (and (not (null? qs))
         (char=? q (car qs))))

  (let loop ([chars (string->list str)]
             [qs '()]
             [res '()])
    (match chars
      ['() (reverse res)]
      ;; Avoid tokenizing string contents.
      [`(,(or #\" #\') ,_ ...) (cond
                                 [(null? qs) (loop (cdr chars)
                                                   (cons (car chars) qs)
                                                   (cons (char->token (car chars)) res))]
                                 [(quote-match? (car chars) qs) (loop (cdr chars)
                                                                      (cdr qs) ; drop opening quote from stack.
                                                                      (cons (char->token (car chars)) res))]
                                 [else (loop (cdr chars) qs (cons (car chars) res))])]
      ;; Drop whitespace if not in string.
      [`(,(? char-ws?) ..1 ,r ...) (if (null? qs) (loop r qs res)
                                       (loop (cdr chars) qs (cons (car chars) res)))]
      [_ (if (null? qs) (loop (cdr chars) qs (cons (char->token (car chars)) res))
             (loop (cdr chars) qs (cons (car chars) res)))])))

(define (print-ast ast)
  (define (print-inner ner)
    (define (print-ws ner)
      (unless (null? (cdr ner))
        (printf " ")))

    (match ner
      ['() (void)]
      [`(,(? pair?) ,_ ...) (print-ast/r (car ner))
                            (print-ws ner)
                            (print-inner (cdr ner))]
      [`(,(kvp k v) ,_ ...) (printf "(~v: " k)
                            (print-inner `(,v))
                            (printf ")")
                            (print-ws ner)
                            (print-inner (cdr ner))]
      [_ (printf "~v" (car ner))
         (print-ws ner)
         (print-inner (cdr ner))]))

  (define (print-ast/r res)
    (match res
      ['() (void)]
      [`(arr (,r ...)) (printf "(arr (")
                       (print-inner r)
                       (printf "))")]
      [`(obj (,r ...)) (printf "(obj (")
                       (print-inner r)
                       (printf "))")]
      [_ (printf "~v" res)]))

  (print-ast/r ast)
  (printf "\n"))

;; Tests
(print-ast (parse-json (tokenize "[12.4, 2,3,, 4,      5]")))
(print-ast (parse-json (tokenize "[1, 2,'ab[c, NaN]ef' , -Infinity, 5, true]")))
(print-ast (parse-json (tokenize "[1, 2,'ab\\\"cd\\\"ef' , 4, 5]")))
(print-ast (parse-json (tokenize "['true\"false\"',, [1,false,  -2.14 ,3] ,88]")))
(print-ast (parse-json (tokenize "[1, 2,'ab[c\\\\d]ef' , {\"Inf'in'ity\": NaN}, ['true', [1,false,   2.148 ,NaN],88], 5, true]")))
(print-ast (parse-json (tokenize "'some_text'")))
(print-ast (parse-json (tokenize "[{'some_key': 14,\"another_key\": [8, 1, 2], 'yet_another_key': {'abc': 123}}, 420,{'wt':false}]")))
Click to expand...
 
Last edited:
y a t s said:
For Go, it's almost certainly something with reflection. It's the closest built-in thing to anything resembling meta-programming in Go imo.
Click to expand...
Yeah, definitely. That would be a big part of whatever code they produce.

Well, I'm wondering how they handle memory management and the generated API.

Like, I would assume they'll support taking a []byte or some buffer type and parsing a struct out of it. That's easy.

But how do they handle having an existing parsed struct and assigning, say, a text field. It's going to involve resizing the underlying buffer and then I'm thinking about if any other code is sharing that same buffer. Just things like that rattling around in my head.

I know that operations like that won't be super efficient and I probably won't even use them. I think just a basic "parse in" and "write out in one go", plus accessors, is probably 99% of what I"ll use this for. But still interesting to think about.

In other use cases (not mine), CapnProto and similar formats can be mmap'd, so some people might actually get up to that sort of thing. But I guess even if you're mmapping it, you'd probably be better off just keeping a separate buffer to compose your object and then doing one final memcpy when you're done.

Edit: Ok, Go capnp has an Arena type for composing capnp structures for reading/writing.
 
Zeftax said:
Reminds me of my first programming job, where I was to rewrite a ~10 kloc VBA script (all in a single function) into C#. Not so fun* times lol, though still better than the development hell that followed...
* Looking back, it was actually pretty funny. A 2D point array was a float array, and you had to make sure to keep the odd/even alignment when reading it, for example. Also a loop, helpfully commented "planes", that was around 5 kloc long, that made me literally hallucinate while trying to understand and rewrite it, the only time a piece of code made me have such an experience.
Click to expand...
Its like being a detective sometimes you have

Oh i wasn't done, anyway...

...put yourself in their shoes and think like them in a non technical way, like what were they thinking. This one guy was clearly doing lots of stuff with some kind of macro recorder thing so you have to think in mouse clicks, the worst was a guy who had a stroke i was always wondering was this pre or post stroke?
 
Last edited by a moderator:
Sveny Mausehund said:
Its like being a detective sometimes you have

Oh i wasn't done, anyway...

...put yourself in their shoes and think like them in a non technical way, like what were they thinking. This one guy was clearly doing lots of stuff with some kind of macro recorder thing so you have to think in mouse clicks, the worst was a guy who had a stroke i was always wondering was this pre or post stroke?
Click to expand...
My dad told me a story of when he was an engineer writing Fortran code, he had to go through and document a piece of code written by someone who had since left, let’s call him Jake. The code was doing some simulations on some part, and all throughout the code there was this one constant that kept cropping up, we’ll call it the Jake factor. It was nowhere to be seen in the actual equations the simulation was supposed to use, but it always seemed to find its way into every function the guy wrote. Eventually he figured it out: whenever the simulation didn’t line up with the expected results, Jake would just adjust the Jake factor until everything lined up.
 
>be me
>use haxe
>compile my code into C++
>expect a significant performance increase compared to JS
>performance stays roughly the same
>dig through generated C++ code
>most functions return Dynamic instead of instances/pointers of classes they should return
>Dynamic's properties are accessed via reflection
>apparently this is expected behavior
cat coping with pain.gif
 
Whatevermancer said:
>most functions return Dynamic instead of instances/pointers of classes they should return
>Dynamic's properties are accessed via reflection
Click to expand...
what do you mean by "return Dynamic"
did it generate code that tries to circumvent the static type checking of c++ or something?
 
DumbDude43 said:
what do you mean by "return Dynamic"
did it generate code that tries to circumvent the static type checking of c++ or something?
Click to expand...
No, I mean it generates code that looks like:
C++: 
Array<Dynamic> findPath(geom::IntPoint start, geom::IntPoint end, Dynamic graph);
when the haxe source looks like this:
Code: 
function findPath(start: IntPoint, end: IntPoint, graph: PathGraph): Array<IntPoint>
Apparently it happens if your class implements an interface(and when you use generics, I guess) cause C++ doesn't have interfaces, it only has classes with virtual methods and implementing multiple interfaces would be turned into multiple inheritance in C++, and the devs just didn't want to deal with that, I guess. There are ways to circumvent this by using abstract classes instead of interfaces and special metadata, but that requires refactoring and is not well documented like basically everything else in haxe.
 
Whatevermancer said:
No, I mean it generates code that looks like:
C++: 
Array<Dynamic> findPath(geom::IntPoint start, geom::IntPoint end, Dynamic graph);
when the haxe source looks like this:
Code: 
function findPath(start: IntPoint, end: IntPoint, graph: PathGraph): Array<IntPoint>
Apparently it happens if your class implements an interface(and when you use generics, I guess) cause C++ doesn't have interfaces, it only has classes with virtual methods and implementing multiple interfaces would be turned into multiple inheritance in C++, and the devs just didn't want to deal with that, I guess. There are ways to circumvent this by using abstract classes instead of interfaces and special metadata, but that requires refactoring and is not well documented like basically everything else in haxe.
Click to expand...
oh so it doesn't properly compile your IntPoint and PathGraph classes into actual c++ classes/structs but instead substitutes them with some generic "Dynamic" class that (i assume) carries additional metadata and is involved in some kind of runtime type checking?
and it does properly compile the IntPoint type when it's a function argument, but not when it's a template parameter?

seems like the haxe -> c++ transpiler just hasn't had a lot of optimisation work done on it
 
DumbDude43 said:
oh so it doesn't properly compile your IntPoint and PathGraph classes into actual c++ classes/structs but instead substitutes them with some generic "Dynamic" class that (i assume) carries additional metadata and is involved in some kind of runtime type checking?
and it does properly compile the IntPoint type when it's a function argument, but not when it's a template parameter?

seems like the haxe -> c++ transpiler just hasn't had a lot of optimisation work done on it
Click to expand...
Basically, yeah.
 
If I'm not given $10M USD by noon Friday, I will write a kernel centered around node so that JS and JS developers infiltrate ring 0.
This IS a credible threat of violence.
 
Anti Snigger said:
If I'm not given $10M USD by noon Friday, I will write a kernel centered around node so that JS and JS developers infiltrate ring 0.
This IS a credible threat of violence.
Click to expand...
A lisp microkernel could be the next TempleOS. I'm sending agents to surreptitiously observe you at the corners of your vision as we speak.
 
Anti Snigger said:
If I'm not given $10M USD by noon Friday, I will write a kernel centered around node so that JS and JS developers infiltrate ring 0.
This IS a credible threat of violence.
Click to expand...
You guys joke, but somewhere in India a jeet is working to make this a reality.
 
Kosher Dill said:
Didn't someone already do this with JVM bytecode? I feel like if India is Mars, that's the moonbase staging ground.
Click to expand...
Sun released hardware that could execute JVM back in the 90s, I don't recall exactly why they stopped
 

I somehow hadn't see this interview with Casey Miratoru before. He goes into the background a bit that led to making a windows terminal that didn't have shit performance.

Then some other technical discussion.

I always enjoy when people interview him. He usually has something interesting to say.
 
Rango Dango said:
the installer for windows is picked up by AVs which i find funny
Click to expand...
This happens a lot with a lot of languages, I believe it has to do with it comparing what it links to with what known virus programs link to which should only be part of virus matching but retards & jeets at Microsoft have different ideas.

prollyanotherlurker said:
He goes into the background a bit that led to making a windows terminal that didn't have shit performance.
Click to expand...
The GitHub issue behind this one is funny but also reminds me we need to kill every single programmer who treats computers as some form of magic and refuses to understand how they work even after basing their whole career around these things.

 
You must log in or register to reply here.
Back
Top Bottom