UltraSynBERT_All_Archives

Decoding the RNA interactome by UltraSynBERTThe official repository for the article "Decoding the RNA interactome by UltraSynBERT".OverviewRNA encodes complex molecular behaviors that underlie binding, regulation, and cellular fate. Current RNA language models have been shaped primarily by endogenous transcripts, leaving the potential of exogenous RNAs to extend model boundaries unexplored. We present UltraSynBERT, an RNA language model pretrained on synthetic RNA binding landscapes to capture RNA binding properties across in vitro and in vivo contexts. Fine-tuned on in vitro SELEX datasets, the model identifies RNA aptamers for diverse targets, including small molecules, proteins, cells, and tissues. The model also predicts tissue specificity for millions of RNA species across 22 human organs based on their 3’-UTR sequences, captures humanpathogenic viral RNA tropism, recognizes RNA base modifications, and characterizes SARS-CoV-2 replicase binding with single-base resolution features. UltraSynBERT Archives1. Pre-training datasets and modelsIn this repository, you will find the following pre-training datasets and pre-trained model checkpoints:Model NameParametersPre-training DataRNA Species NumberDescriptionCheckpointUltraSynBERT33.5MUltraSelex SiR-B10 millionpre-training from scratchUltraSynBERT.pklUltraSynBERTsource_SiR33.5MSELEX SiR10 millionpre-training from scratchUltraSynBERT_source_SiR.pklUltraSynBERTsource-nsp1233.5MSELEX nsp1210 millionpre-training from scratchUltraSynBERT_source_nsp12.pklUltraSynBERTmolecules33.5M12 SELEX targets’ training sets3.24 millioncontinued pre-training the basic UltraSynBERT modelUltraSynBERT_molecule.pklUltraSynBERT3UTR33.5MThe preliminary 3’-UTR full training datasets1.69 millioncontinued pre-training the basic UltraSynBERT modelUltraSynBERT_3UTR.pklUltraSynBERTplus33.5M12 SELEX targets’ training sets and the preliminary 3’-UTR full training datasets4.93 millioncontinued pre-training the basic UltraSynBERT modelUltraSynBERT_plus.pklUltraSynBERTRIP33.5MHuman RIP-Seq in ENCODE6.88 millioncontinued pre-training the basic UltraSynBERT modelUltraSynBERT_RIP.pklNOTE:The pretraining data is located in the Dataset Availability\1. Dataset for pre-training folder.The pretrained models can be found in the Model Availability\1. Pre-training models and Model Availability\2. Continued pre-training models folders. 2. Downstream task datasets and fine-tuned modelsAdditionally, this official repository also provides all datasets for downstream tasks and supervised fine-tuned models based on the base UltraSynBERT for downstream applications, including: RNA aptamer identification models targeting small molecules, proteins, cells, and tissues (12 specialized datasets), and tissue-specificity prediction for human mRNA 3'-UTR regions.Task TypeData TypeDataset NumberDatasetSiR-binding predictionIn vitro3(1) UltraSelex-SiR-B.csv (2) SELEX-SiR-whole.csv (3) SELEX-SiR-exclusive.csvIn vitro RNA-target binding predictionIn vitro12(1) SELEX-small-molecule-DAse.csv; (2) SELEX-small-molecule-BC.csv: (3) SELEX-small-molecule-PR.csv (4) SELEX-small-molecule-MI.csv: (5) SELEX-protein-TARDBP.csv: (6) SELEX-protein-RT.csv (7) SELEX-protein-RBM24.csv; (8) SELEX-protein-S15.csv; (9) SELEX-(multi)cellular-ISLETS.csv (10) SELEX-(multi)cellular-MDSC.csv; (11) SELEX-(multi)cellular-CHO-K1.csv;(12) SELEX-(multi)cellular-TNBC.csvIn vivo RNA-protein interactions prediction (human source)In vivo11(1) 16_ICLIP_hnRNPC_Hela_iCLIP_all_clusters_sequences.csv; (2) 17_ICLIP_HNRNPC_hg19_sequences.csv;(3) 18_ICLIP_hnRNPL_Hela_group_3975_all-hnRNPL-Hela-hg19_sum_G_hg19--ensembl59_from_2337-2339-741_bedGraph-cDNA-hits-in-genome_sequences.csv (4) 19_ICLIP_hnRNPL_U266_group_3986_all-hnRNPL-U266-hg19_sum_G_hg19--ensembl59_from_2485_bedGraph-cDNA-hits-in-genome_sequences.csv; (5) 20_ICLIP_hnRNPlike_U266_group_4000_all-hnRNPLlike-U266-hg19_sum_G_hg19--ensembl59_from_2342-2486_bedGraph-cDNA-hits-in-genome_sequences.csv; (6) 22_ICLIP_NSUN2_293_group_4007_all-NSUN2-293-hg19_sum_G_hg19--ensembl59_from_3137-3202_bedGraph-cDNA-hits-in-genome_sequences.csv (7) 27_ICLIP_TDP43_hg19_sequences.csv; (8) 28_ICLIP_TIA1_hg19_sequences.csv; (9) 29_ICLIP_TIAL1_hg19_sequences.csv (10) 30_ICLIP_U2AF65_Hela_iCLIP_ctrl_all_clusters_sequences.csv; (11) 31_ICLIP_U2AF65_Hela_iCLIP_ctrl+kd_all_clusters_sequences.csvIn vivo RNA-protein interactions prediction (mouse source)In vivo11(1) CLIP-mouse-EZH2-sequences.csv; (2) CLIP-mouse-FUS-sequences.csv; (3) CLIP-mouse-HNRNPR-sequences.csv (4) CLIP-mouse-LIN28A-sequences.csv; (5) CLIP-mouse-RBFOX2-sequences.csv; (6) CLIP-mouse-RBM10-sequences.csv (7) CLIP-mouse-SRSF2-sequences.csv; (8) CLIP-mouse-SRSF3-sequences.csv; (9) CLIP-mouse-TARDBP-sequences.csv (10) CLIP-mouse-U2AF2-sequences.csv; (11) CLIP-mouse-YTHDC2-sequences.csvmodification In vivo13m6A: (1) m6A-A549.csv; (2) m6A-CD8T.csv; (3) m6A-ESC.csv (4) m6A-HCT116.csv; (5) m6A-HEK293.csv; (6) m6A-HEK293T.csv (7) m6A-Hela.csv; (8) m6A-HepG2.csv; (9) m6A-MOLM13.csvm1A: m1A_dataset_with_split.csvm5C: m5c_dataset_with_split.csvm6Am: m6Am_dataset_with_split.csvpseudouridine: pseudouridine_dataset_with_split.csv3'UTR tissue-specific recognitionIn vivo1human_22_tissue_three_terminal_UTR.csvZero-shot RNA mutation effect prediction targeting SARs-CoV-2 replicase nsp12In vitroThe wild-type sequence and seven mutated sequencesRNA-mutation.fastaRNA post-transcriptional regulation–related tasks in vitro/In vivo4(1) miRNA _interactions.csv; (2) mRNA_subcellular_localization.csv; (3) Polyadenylation_signal.csv; (4) RNA_stability.csv human-pathogenic RNA virusesIn vivo1virus.csvNOTE:The datasets for downstream tasks are in the Dataset Availability\2. Dataset for fine-tuning folder.The fine-tuned models for downstream tasks are available in the Model Availability\3. Fine-tuned models folder.