targon
1.0.0
Targon (Bittensor Subnet 4) is a deterministic verification mechanism that is used to incentivize miners to run openai compliant endpoints and serve synthetic and organic queries.
NOTICE: Using this software, you must agree to the Terms and Agreements provided in the terms and conditions document. By downloading and running this software, you implicitly agree to these terms and conditions.
For validators we recommend a 8xA100, although a 1xA100 could also be used. We plan on focusing on bringing these costs down in the coming updates.
For miners, A100 or H100s are common choices. Benchmarking is up to the miner to determine what GPU works best for their optimizations.
In order to run Targon, you will need to install PM2 and the Targon package. The following instructions apply only to Ubuntu OSes. For your specific OS, please refer to the official documentation.
To install or update nvm, you should run the install script. To do that, you may either download and run the script manually, or use the following cURL or Wget command:
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bashRunning either of the above commands downloads a script and runs it. The script
clones the nvm repository to /.nvm, and attempts to add the source lines from
the snippet below to the correct profile file (/.bash_profile, ~/.zshrc,
~/.profile, or ~/.bashrc).
export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvmnvm install nodenpm install pm2@latest -gYou have now installed PM2.
git clone https://github.com/manifold-inc/targon.git
cd targonpython3 -m pip install -e .You have now installed Targon. You can now run a validator or a miner.
Before starting or registering your miner in Targon, first you will want to run VLLM serving different images validators are requesting. You can find a list at https://stats.sybil.com/stats/validator under the live tab. The more models you run, the higher your incentive.
VLLM is the recommended engine, however it is not required. If you are using
VLLM, make sure yo include the --return-tokens-as-token-ids flag, or else your
responses will fail.
Once you have one (or multiple) models running, modify the default miner code to
proxy to the proper VLLM instance on each request. Verifiers will include the
X-Targon-Model header so that the miner node does not need to parse the actual
body.
In the miner.py script you will find a function called list_models. To serve multiple models you must:
async def list_models(self):
return [
"ExampleName/Meta-Llama-3.1-8B-Instruct",
"ExampleName/mythomax-l2-13b",
"ExampleName/Hermes-3-Llama-3.1-8B",
"ExampleName/Nxcode-CQ-7B-orpo",
"ExampleName/deepseek-coder-33b-instruct",
"ExampleName/Llama-3.1-Nemotron-70B-Instruct-HF",
]
create_chat_completion and create_completion methods in neurons/miner.py to route to the appropriate vllm upstream server based on the model (which is either in the headers or from the request payload's model param)Here is a hint / incomplete code snippet to get you started:
model_port_map = {
'ExampleName/mythomax-l2-13b': 1001,
'ExampleName/Hermes-3-Llama-3.1-8B': 1002,
'ExampleName/Nxcode-CQ-7B-orpo': 1003,
'ExampleName/deepseek-coder-33b-instruct': 1004,
'ExampleName/Llama-3.1-Nemotron-70B-Instruct-HF': 1005
}
full_url = f"http://127.0.0.1:{model_port_map.get(body.get('model'), 1000)}{path}"
Once this is complete, you are ready to continue starting your miner node.
Running a miner through PM2 will require the vLLM instance to be running.
pm2 start neurons/miner.py --name miner --interpreter python3 -- --wallet.name [WALLET_NAME] --netuid 4 --wallet.hotkey [WALLET_HOTKEY] --subtensor.network finney --model-endpoint [MODEL_ENDPOINT] --api_key [API_KEY] --axon.port [AXON PORT] --logging.tracePlease replace the following with your specific configuration:
- [WALLET_NAME]
- [WALLET_HOTKEY]
- [MODEL_ENDPOINT]
- [API_KEY]
- [AXON_PORT]
NOTE: Trace logging is very verbose. You can use --logging.info instead for
less log bloat.
Additionally:
--no-force-validator-permit [TRUE/FALSE]
is defaulted to false to force incoming requests to have a permit. Set this to true if you are having trouble getting requests from validators on the 'test' network.
Validators are simply run through pm2, enabling auto restarts and auto updates. A validator should be run on atleast an A100, but the larger the better, as larger clusters can handle more models. The machine should have nvidia-smi / cuda installed along with docker.
No vllm instance needed
Validator Instance:
pm2 start neurons/validator.py --name validator --interperter python3 -- --wallet.name [WALLET_NAME]
Please replace the following with your specific configuration:
- [WALLET_NAME]
default, or config.
endpoint: defaults to https://targon.sybil.com/api/models. This will
mimic the manifold validatordefault: only run NousResearch/Meta-Llama-3.1-8B-Instructconfig: parse a text file named models.txt with a list of models
separated by newlinesendpoint. Sets the
api endpoint to ping for list of models. Defaults to targon hub.Example model config file
models.txtNousResearch/Meta-Llama-3.1-8B-Instruct NousResearch/Meta-Llama-3.1-70B-Instruct NousResearch/Meta-Llama-3.1-405B-Instruct
Autoupdate is implemented in targon/utils.py. This is to ensure that your codebase matches the latest version on Main of the Targon Github Repository.
Validator Autoupdate is implemented and defaulted to run once weights have been set. To disable, please add the flag to your command line build:
pm2 start neurons/validator.py --name validator --interperter python3 -- --wallet.name [WALLET_NAME] --autoupdate-offMiner Autoupdate is not implemented. Miners will need to check the Targon repository and update themselves as new versions are released. If interested in utilizing the autoupdate feature that Validators use, please follow the steps below:
NOTE: This will not be maintained by the Manifold Labs Team.
from targon.updater import autoupdate if self.config.autoupdate:
autoupdate(branch="main")The goal of the hub is to give validators a simple way to directly generate revenue off of their bittensor bandwidth. This is designed as a template for validators to take and create their own branded hubs with, however pull requests are still encouraged.
If you are interested in running your own instance of Targon Hub, you will need to add an additional flag to save the records of miners' responses to a PostgreSQL DB.
NOTE: No flag means no database!
--database.url [DB_CONNECTION_STRING]
Please replace the following with your specific connection URL:
- [DB_CONNECTION_STRING]
Below are steps to create a Supabase connection string to utilze this feature:
Connect button near
the top right of the screentransaction to session in the dropdown--database.url flag and connection stringPlease reach out to the SN4 team for help setting up targon hub in sn4 chat or our discord
pm2 start neurons/validator.py --name validator --interperter python3 -- --wallet.name [WALLET_NAME] --database.url [DB_CONNECTION_STRING]As your validator runs, you will start seeing records being added into your Supabase database. This will be directly what your Targon Hub will query.
Targon uses a novel, in-house proof-of-work value to verify model responses. These are generated just like log-probs, and can be used to verify with high accuracy that a response was generated with the model requested. We use this in combination with a fast and slow version of log-prob verification. Verified responses are scored by speed and consistency of verification.
A miner is a node that is responsible for generating a output from a query, both organic and synthetic.
A validator is a node that is responsible for verifying a miner's output. The validator will send an openai compliant request to a miner with. The miner will then send back a response with the output. The validator will then use the proof of work values of the response to verify that each miners response is accurate. Validators will keep score of each miners response time and use their averages to assign scores each epoch.
Validators can query miners directly using any openai package, and Epistula headers. Below is boilerplate for querying a miner in python.
miner = openai.AsyncOpenAI(
base_url=f"http://{axon.ip}:{axon.port}/v1",
api_key="sn4",
max_retries=0,
timeout=Timeout(12, connect=5, read=5),
http_client=openai.DefaultAsyncHttpxClient(
event_hooks={
"request": [
# This injects Epistula headers right before the request is sent.
# wallet.hotkey is the public / private keypair
#
# You can find this function in the `epistula.py` file in
# the targon repo
create_header_hook(wallet.hotkey, axon.hotkey_ss58)
]
}
),
)Project maintainers reserve the right to weigh the opinions of peer reviewers using common sense judgement and may also weigh based on merit. Reviewers that have demonstrated a deeper commitment and understanding of the project over time or who have clear domain expertise may naturally have more weight, as one would expect in all walks of life. Where a patch set affects consensus-critical code, the bar will be much higher in terms of discussion and peer review requirements, keeping in mind that mistakes could be very costly to the wider community. This includes refactoring of consensus-critical code. Where a patch set proposes to change the Targon subnet, it must have been discussed extensively on the discord server and other channels, be accompanied by a widely discussed BIP and have a generally widely perceived technical consensus of being a worthwhile change based on the judgement of the maintainers. That being said, Manifold welcomes all PR's for the betterment of the subnet and Bittensor as a whole. We are striving for improvement at every interval and believe through open communication and sharing of ideas will success be attainable.
