JiRack GPT-2 Initial Weights

This file is strictly intended for saving the initial weights (checkpoint) of the JiRack GPT model.
The model is "clean": it contains no data and has never undergone any pre-training.

It is engineered to be a maximally safe and robust base for training from scratch for specialized, smaller models, such as:

SPAM Detection Systems
FRAUD Detection Models
Background Check (BG Check) Models

A product of CMS Manhattan.

Tokenizer Choices

For English: GPT-2 Hugging Face tokenizer
For multilingual use: BERT tokenizer from the Hugging Face library

Model Architecture Details

GPT-2 Architecture (Classic, Transformer-like)

CustomEmbedding
FrozenSignatureLayer
LearnedPositionalEmbedding
[TransformerBlock]
    ├── MultiHeadAttention
    ├── LayerNorm
    ├── LayerNorm
    ├── FFN
          ├── Linear
          ├── Activation: GELU
          └── Linear
LayerNorm
Linear

Model Checkpoint File Explanations

12-head Attention Model

Parameters:

VOCAB_SIZE = 50257
MODEL_DIM = 768
NUM_HEADS = 12
NUM_LAYERS = 6
MAX_SEQ_LEN = 8192
FFN_HIDDEN_DIM = 4 * MODEL_DIM
HEAD_DIM = MODEL_DIM // NUM_HEADS

File:
JiRack_H12_L6_V50257_D768_MSL8192_FF768x4.pt

6-head Attention Model

Parameters:

VOCAB_SIZE = 50257
MODEL_DIM = 768
NUM_HEADS = 6
NUM_LAYERS = 6
MAX_SEQ_LEN = 8192
FFN_HIDDEN_DIM = 4 * MODEL_DIM
HEAD_DIM = MODEL_DIM // NUM_HEADS

File:
JiRack_H6_L6_V50257_D768_MSL8192_FF768x4.pt

So About PyTorch script . You can use Pytorch script for AI classification task .
Do not Jit for Chatbot task . Use just state dict PyTorch for GPT (Chatbot) tasks

See other models with same patterns for read parameters

install tokenizer before run

mkdir -p tokenizer
wget -O tokenizer/tokenizer.json https://huggingface.co/gpt2/resolve/main/tokenizer.json
wget -O tokenizer/vocab.json https://huggingface.co/gpt2/resolve/main/vocab.json
wget -O tokenizer/merges.txt https://huggingface.co/gpt2/resolve/main/merges.txt
wget -O tokenizer/tokenizer_config.json https://huggingface.co/gpt2/resolve/main/tokenizer_config.json

JiRack RAG System

It is microservice architecture with API Gateway and Service Discovery
Framework Spring boot and Google embeddings model for JiRack RAG System with Chatbot and JiRach model deployment with docker scipt
video https://www.youtube.com/watch?v=vHClQu76kMc
RAG System https://bitbucket.org/cmsmanhattan/rag/src/main/

Copyright Office

From:
[email protected]
To:
[email protected]
Mon, Dec 15 at 7:31 AM
THIS IS AN AUTOMATED EMAIL. PLEASE DO NOT REPLY.
Thank you for submitting your registration claim using the Electronic Copyright Office (ECO) System.
The following files were successfully uploaded for service request 1-15058193231
File Name :jirack_gpt2_class_pytorch.zip
File Size :2993 KB
Date/Time :12/15/2025 7:27:48 AM
[THREAD ID: 1-6X1C895]
United States Copyright Office

Welcome to ask to design your corp model over 33B or 70B or more parameters

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 1 Ask for provider support