• 1 Post
  • 21 Comments
Joined 2 years ago
cake
Cake day: August 3rd, 2023

help-circle



  • I wasn’t attempting to attack what you said, merely pointing out that once you cross the line into philosophy things get really murky really fast.

    You assert that LLMs aren’t taught the rules, but every word is not just a word. The tokenization process includes part of speech tagging, predicate tagging, etc. The ‘rules’ that you are talking about are actually encapsulated in the tokenization process. The way the tokenization process for LLMs, at least as of a few years ago when I read a textbook on building LLMs, is predicated on the rules of the language. Parts of speech, syntax information, word commonality, etc. are all major parts of the ingestion process before training is done. They may not have had a teacher giving them the ‘rules’, but that does not mean it was not included in the training.

    And circling back to the philosophical question of what it means to “learn” or “know” something, you actually exhibited what I was talking about in your response on the math question. Putting to piles of apples on a table and counting them to find the total is a naïve application of the principals of addition to a situation, but it is not describing why addition operates the way it does. That answer does not get discussed until Number Theory in upper division math courses in college. If you have never taken that course or studied Number Theory independently, you do not know ‘why’ adding two numbers together gives you the total, you know ‘that’ adding two numbers together gives you the total, and that is enough for your life.

    Learning, and by extension knowledge, have many forms and processes that certainly do not look the same by comparison. Learning as a child is unrecognizable when compared directly to learning as an adult, especially in our society. Non-sapient animals all learn and have knowledge, but the processes for it are unintelligible to most people, save those who study animal intelligence. So to say the LLM does or does not “know” anything is to assert that their “knowing” or “learning” will be recognizable and intelligible to the lay man. Yes, I know that it is based on statistical mechanics, I studied those in my BS for Applied Mathematics. I know it is selecting the most likely word to follow what has been generated. The thing is, I recognize that I am doing exactly the same process right now, typing this message. I am deciding what sequence of words and tones of language will be approachable and relatable while still conveying the argument I wish to levy. Did I fail? Most certainly. I’m a pedantic neurodivergent piece of shit having a spirited discussion online, I am bound to fail because I know nothing about my audience aside from the prompt to which you gave me to respond. So I pose the question, when behaviors are symmetric, and outcomes are similar, how can an attribute be applied to one but not the other?





  • See, I agree with everything up to the end. There you are getting into the philosophy of cognition. How do humans answer a question? I would argue, for many, the answer for most topics would be "I am repeating what I was taught/learned/read. An argument could be made that your description of responding with “What would a realistic answer to this question look like?” is fundamentally symmetric with “This is what I was taught.” Both are regurgitating information fed to them by someone who presumably (hopefully) actually had a firm understanding of the material themselves. As an example: we are all taught that 2+2=4, but most people are not taught WHY 2+2=4. Even fewer are taught that 2+2=11 in base 3 or how to convert bases at all. So do people “know” that 2+2=4 or are they just repeating the answer that they were told was correct?

    I am not saying that LLMs understand or know anything, I am saying that most humans don’t either for most topics.


  • Just letting you know that you are definitely suffering a Dunning Kruger moment. I have had to study how printers work all the way down to the electrochemical level and no paper printer is simple to make aside from a dot matrix printer. In my educated opinion I will say that you are borderline correct that a laser printer might be more feasible from a materials standpoint, neither are feasible in general for a home lab, let alone a guy at his kitchen table assembling a kit.

    Laser printers and ink jet printers involve extremely complex electrochemical and physical processes to function that a home lab is going to struggle to replicate. The optics for laser printers have to be assembled in a clean room because even a single stray particle of dust will destroy the print quality for a large section of the page. Ink jet printers use tiny heaters to vaporize the ink and electric fields to propel it through a nozzle to the page. These jets are created using multilayer acid etched circuitboards that are precicely tuned for the specific inks and substrates that are used.

    These are just examples of challenges to overcome, and are by no means exhaustive.

    I hate the printer companies probably more than anyone here, so do not think I am a shill for them. I am articulating why what you are saying is impractical and unreasonable. Laser printers are probably easier to source most of the mechanical parts, but assembly is obscenely delicate and prohibitive. Ink jet printers are simpler to recreate and assemble, but the miniaturization required makes it impossible to do at home.


  • You are missing a vital part of how a laser printer works. The toner is dusted onto a drum that has a static charge. That charge is manipulated by the laser, which means that there is a very specific frequency that the laser has to utilize and it has to be keyed to the material used for the drum. I would have to dig more into the specific interaction, but I am pretty sure that off the rack lasers and drums are not going to be functional, both in wattage and frequency.

    So the laser printer process is: a laser traces the negative space on a drum with a static charge to discharge those spots, next a pigment substrate is dusted onto the drum, being held by the static where the positive space is going to be. Following that, a heater heats the substrate to permanently affix it to the page.


  • I have never used an AI to code and don’t care about being able to do it to the point that I have disabled the buttons that Microsoft crammed into VS Code.

    That said, I do think a better use of AI might be to prepare PRs in logical and reasonable sizes for submission that have coherent contextualization and scope. That way when some dingbat vibe codes their way into a circle jerk that simultaneously crashes from dual memory access and doxxes the entire user base, finding issues is easier to spread out and easier to educate them on why vibe coding is boneheaded.

    I developed for the VFX industry and I see the whole vibe coding thing as akin to storyboards or previs. Those are fast and (often) sloppy representations of the final production which can be used to quickly communicate a concept without massive investment. I see the similarities in this, a vibe code job is sloppy, sometimes incomprehensible, but the finished product could give someone who knew what the fuck they are doing a springboard to write it correctly. So do what the film industry does: keep your previs guys in the basement, feed them occasionally, and tell them to go home when the real work starts. (No shade to previs/SB artists, it is a real craft and vital for the film industry as a whole. I am being flippant about you for commedic effect. Love you guys.)






  • Indeed. I agree with everything you said here.

    Most of my quandary stems from patants at this point I guess. Copyright reform advocates are plentiful, but patant reform is much more rarely mentioned and IMHO is a bigger issue for progress and development of society. The anticompetitive practice of purchasing patants so you can bury them gets deep under my skin. There are so many things that have been invented, problems that have been solved, potential progress that has had the first steps made, that was squelched because some person/company with more money than civic duty realized that it would negatively affect their revenue stream. And instead of developing the idea and incorporating it to make their own products better, they just hide if in a vault somewhere.

    That all said, I cannot describe how happy I was when I heard of some rogue patant whore activists out there coming up with ideas for enshittification and patanting them so corpos cannot use those specific methods to enshittify our world more. I wish I had been able to patant the SaaS architecture when I graduated HS in 2003. Maybe the world would be a much better place.






  • Unpopular opinion around here, but I feel that there is a place for IP in the world. Yes, it is a flawed system that is abused by corpos, but it is also a system that can and does protect the work of the little guy. Copyleft has a place, FOSS has a place, and copyright, needs reformed.

    DMCA should never have been signed into law the way it was. It limited “free use” way too much and is now being weaponized by police to shirk accountability. Corporations abuse it by buying up technology which would compete with them and burying it. Etc.

    My fix:

    • Repeal DMCA. Full stop.
    • Place a requirement on patant/IP purchases that they have to be a material component of a product brought to market within 5 years of purchase else the ownership of the patant/IP reverts to its original creator with no recompense to the purchaser.
    • No transfer of ownership of purchased IP can be made without informing the original creator and giving a reasonable period for them to object. If they object, they must have the right to file the complaint in court to reassert ownership, which may involve reasonable recompense as decided by the court. Also, all clauses in current contracts related to transfer of ownership become void and unenforceable.
    • Any purchases made of patants or technical IP must be commensurate with the market cap of the highest level owner in the subsidiary chain. Nestlè does not get to buy the design for a new water filtration under some barely visible water brand to pretend like they cannot afford to pay what it is worth.
    • Not specific, but still relevant, to patant/IP situations: Forced arbitration becomes illegal and unenforceable. Everyone has the right to demand their case go before, and be adjudicated by, an impartial and uninvolved judge.

    I have had other reforms, but they are not coming to mind currently. I know it is all a very unpopular opinion around here, but I am personally an independent developer and I want my tools and the code I designed to be used for the purposes I have designed them for, and I don’t want someone lifting algorithms I invented and not giving credit or licensing it from me. I am one man who has a family that he struggles to feed, and I recognize that the copyright and patant protections are, ostensibly, there to protect my work as well.